(updated 8.10.2020)

8.10.2020 12:55pm - I feel like a dinosaur - A 30 yr old paper of ours just got a "Classic Paper Award" from the AI Journal. The paper deals with temporal constraints, a hot topic in the 1980's. To see why, go to Example 1.1 here:

8.10.2020 5:39am - (1/ ) At first I thought that this paper, on Predictive Model Interpretation via Causal Attribution violates the Ladder of Causation, because "attribution" is not definable at Rung-1. Now I see that the authors define "attribution" differently,
8.10.2020 5:39am - (2/2) taking the data-fitted function as a new data generation process and defining "attribution" with respect to this environment. Not sure how problematic this substitution can be. For example, attributing diseases to symptoms, and crimes to ice-cream sales.

8.10.2020 5:14am - For economists immersed in the time-series tradition, this paper by Ken Hoover provides an interpretation of Granger Causality, cointegration, causal-order and other esoteric concepts in terms of DAGs.

8.10.2020 2:50am - (Replying to @Rob_Malley) And with an answer like this @Rob_Malley wants to be taken seriously. "All means all": the elephant and the mouse, rich and poor, merchant & chief. And there is no hidden agenda behind this charitable egalitarian analysis of a scene to which he claims expertise. Go on, Nasrallah.

8.9.2020 10:53pm - I would support this proposal only if Statistics is supplemented with causal reasoning, else we are back to "black box" data-fitting mentality, which always comes at the expense of "data-interpretation" - the main focus of data science.

8.9.2020 1:28pm - (Replying to @shriyamite) The paper that comes to mind on application of do() to legal problem is "Causes of effects and effect of causes" Most judicial question deal with individual-level causation, hence, counterfactual logic.

8.9.2020 7:19am - I've seen it with gullible left-liberal journalists. They get invited to smooch with other l-liberal journalists and, forget facts and morals, they fall for the theory that whatever Israel does right it does to deflect attenti

8.9.2020 3:46am - Moreover, given that "one-state" dreamers (eg @Beinart ) know perfectly well that Israelis won't give up their sovereignty no matter what, is it really inappropriate to call their campaign "incitement to genocide"? and their megaphones (eg @NYT ) "accomplices"???

8.8.2020 6:03pm - And it is anti-Zionism (ie Zionophobia) that is becoming the ugliest form of thuggery on US campuses:, in the best tradition of morally bankrupt ideologies.

8.8.2020 5:18pm - We had a tweeter discussion on whether it is ethical, given that reviewers assume a limited audience. I believe the consensus was: it is ethical. I further suggested that historians will find this material more representative of contemporary science than the paper itself.

8.8.2020 3:43am - (Replying to @l_gervasi) I am not familiar with KPI systems. What is the input and what is the output?

8.8.2020 3:38am - (Replying to @abdallah_fayed) I presume you meant "equally indigenous people" and that you have read the facts about the heart of the Zionist program. I happened to research it thoroughly:

8.7.2020 11:58pm - At long last, a major newspaper is calling on University administrators to address the ugliest and least acknowledged form of racism on US campuses -- Zionophobia -- Denying Jews the right to a homeland:

8.7.2020 8:59pm - Clarifying the night-driving paradigm: To make sensible choices when you doubt the structure of your DAG, first learn how to make choices for a given DAG, second, see if your doubts make a difference, finally, see if there is a choice that remains satisfactory despite your doubts

8.7.2020 8:33am - (Replying to @PophamFrank @frankdevocht and 5 others) On the relationships between DAGs and PO see: * Judea Pearl Interview by David Hand (on blog):

8.7.2020 8:27am - (Replying to @PWGTennant @frankdevocht and 6 others) "quasi-experiments" are just observational studies with supposedly credible knowledge about the causal relationships among certain variables.

8.7.2020 12:36am - (Replying to @l_gervasi) Curious what sort of "causal model" you have in mind, that would allow you to decide which of two sets of measurements you should send your patient to.

8.6.2020 11:29pm - A prerequisite for good night driving is knowing how to drive at day time, then compensate for darkness and other agnostic impediments. Using darkness as justification for quitting driving school is awkward.

8.6.2020 10:24pm - A very disturbing incident at USC What's more disturbing, the resigning student complains about "anti-Zionism" (8 times in her letter) and the President of USC apologizes for "antisemitism"-- the easy way out and the obvious way to ensure zero result.

8.6.2020 8:37pm - Controlling for a "broader set" makes sense when measurments have zero cost. But in most real life situations one cannot afford to control for EVERYHING that can possibly be measured and, then, cost-sufficiency-efficiency considerations demand some domain models, namely DAGs.

8.6.2020 6:03pm - And it is anti-Zionism (ie Zionophobia) that is becoming the ugliest form of thuggery on US campuses:, in the best tradition of morally bankrupt ideologies.

8.6.2020 5:18pm - We had a tweeter discussion on whether it is ethical, given that reviewers assume a limited audience. I believe the consensus was: it is ethical. I further suggested that historians will find this material more representative of contemporary science than the paper itself.

8.6.2020 4:42pm - If by "out of sample" one means "under data shift" or "under process shift" than a charitable interpretation can be constructed. Otherwise, @_MiguelHernan is right; the Ladder of Causation has its laws, and "nobody is perfect."

8.6.2020 4:31pm - (Replying to @JPedro_Monteiro @NateSilver538 and @mattyglesias) @NateSilver538 is a great predictor; I don't want to embarrass him when he is wrong. It ain't fair, if he has not read #Bookofwhy.

8.6.2020 4:04pm - I love this article @ShMMor , with one exception -- it does not name the offender. "Anti-Zionism" sounds like some sort of a benign political opinion, socialism, Maoism, anti-liberalism, etc. Zionophobia better expresses the racist and moral deformity of the Anti-Zionist agenda.

8.6.2020 9:54am - (Replying to @ScienceandLove) Why should we "put things philosophically" when we can "put things computationally"? Philosophers are moving from hand waving to computational models, why should we go backward?

8.6.2020 1:36am - Sharing a video of my talk at the CI-FAR workshop on causality: "Contesting the Soul of Data Science"
Apologizing for the occasional mis-synchronization with the slides (which I could not see in LA), due to a mischievous Zoom problem.

8.5.2020 9:54pm - Sharing an uplifting project: Mark O'connor Summer Music Camp, is one of the most meaningful memorial to our son Daniel, his love for music and his hopes for humanity.

8.5.2020 8:02pm - One unexpected consequence of Covid-19 is the unprecedented rise of public interest in reasoning under uncertainty. Here is another NYT story on Bayesian stat: . Their next story is, perhaps, "Why I am only Half Bayesian"

8.5.2020 3:08pm - Remember my post on machine learning and whether all knowledge comes from sense data? Well, in the wake of the CI-FAR workshop on causality, Yoshua Bengio sent me a couple of comments which I have since appended to the post. Enjoy, and join the dance.

8.5.2020 4:55am - (1/ ) Following our (re)discovery of the beautiful paper by White and Lu, here is another forgotten paper, this time for ML folks interested in mining "invariants", as in recent works of Y. Bengio. The forgotten 2001 paper, "Causal Discovery From Changes"
8.5.2020 4:55am - (2/ ) uses "shocks", or spontaneous local changes in the environment ("Nature's interventions") to infer causal directionality among variables. As is common in CI analyses, the paper unveils those features of the world that would "guarantee" correct discovery and those that won't.
8.5.2020 4:55am - (3/3) A more complete and general analysis of this problem is provided by Jaber etal's "Causal Discovery from Soft Interventions with Unknown Targets" (2020) Again, the emphasis is on the theoretical distinction between the "doable" and the "undoable".

8.5.2020 2:36am - (Replying to @IgnaGB26) None in mainstream Stat. Which is both a shame and an opportunity for an author to be "the first". But Primer would easily be handled by stat students/instructor. See

8.4.2020 6:55am - Thank you, America! Exactly sixty years ago, August 4, 1960, me and my wife landed in NYC as graduate students, seeking education and something meaningful to do in the land of unlimited possibilities. Today, looking back, we are grateful for the opportunity. Thank you, America.

8.4.2020 6:51am - (Replying to @eddericu) Yes, see Causality, page 81-82.

8.3.2020 6:34pm - (Replying to @orlipeter @ConceptualJames and @kareem_carr) Sorry, I took @kareem_carr post to be about science (I still think it was) and I suddenly find myself embroiled in an anti-culture war. I hope no one accuses me of taking sides.

8.3.2020 7:06am - Nice discussion and illustration of c-component identification strategy.

8.3.2020 4:20am - (Replying to @jenhelenmar and @trishgreenhalgh) I never understood what "evidence based" medicine is all about, so I feel unqualified to participate in this debate. Perhaps it would jolt "evidence based" philosophers (eg. Cartwright et al) to define what it is in causal terms.

8.2.2020 9:56pm - (Replying to @Isaac_Herzog and @Sethrogen) I don't buy it. @Sethrogen has all the microphones in the world, 10 times more than he had before making that idiotic "jest". He can stand up like a mench and tell the microphones what Israel really means to him. I have not heard him yet. I am listening.

8.2.2020 5:01pm - (Replying to @RaulMachadoG) Attempts? I know of only one attempt: causal diagrams. Would love to hear of alternatives.

8.2.2020 4:58pm - Every "approach" that is based on Rubin's potential outcome "framework" is guilty of bypassing graph construction and of mixing right and wrong. see and and In SCM, non-linearity is assumed a-priori.

8.2.2020 2:46pm - (Replying to @Jabaluck @causalinf and 8 others) The W&L paper cited is saying, if we commit to a class of DAGs with certain features, we get improved efficiency. To communicate what those features are, requires DAG language, eg. drivers, proxies, colliders, chains, etc. Often, a single dag suffices to encode those features.

8.2.2020 11:56am - (1/ ) It is for these reasons I consider myself lucky to be in computer science, where math is regarded as a man-made language in the service of thought, not the other way around. When G. Boole wanted math to serve logic he decide that 1+1=1. When S. Wright wanted math to serve
8.2.2020 11:56am - (2/2) causality, he invented diagrams. If you are still a statistician, ask yourself when the last time was that statistics has switched languages to cope with new type of problems.

8.2.2020 11:34am - (Replying to @Jabaluck @causalinf and 8 others) Sure, observe all the models shown in:; each summarizes trillions of models sharing certain structural features. The point is that you won't know whether OLS delivers causal effects unless you assume those structural features to hold.

8.2.2020 7:00am - I thank @eliasbareinboim for sharing his talk's video at Microsoft Research: . It explains the fundamental challenge of CI in a slightly different way from the way I (and the #Bookofwhy) present the Ladder of Causation.

8.2.2020 6:33am - (1/ ) Dearth of Econ examples could not possibly be the obstacle. If by "traditional means" you mean: "assuming conditional ignorability" then there were hundreds of Econ examples that started that way and ended of course with the perennial bewilderment whether the assumption holds
8.2.2020 6:33am - (2/2) for the selected covariates. Each of these examples could have benefited from the transparency of graphical models, especially in regard to efficiency considerations. (See "A Crash Course in Good and Bad Controls" So what was & still is the obstacle?

8.2.2020 3:21am - Sure. This paper by S. Greenland, J. Pearl, and J.M. Robins, "Causal Diagrams for Epidemiologic Research," has inspired 2638 citations, compared with 41 citations of White and Lu (2011). The shrill silence of dogma.

8.1.2020 7:19pm - (1/ ) I've just (re)discovered a beautiful paper by White and Lu It gives economists a simple introduction to graphical models plus compelling arguments why they're useful, if not necessary. My question to enlightened economists: How come this paper all but
8.1.2020 7:19pm - (2/ ) disappeared from the econometrics literature (only 41 citations). How come it failed to convince econometrics leadership, from Heckman to Angrist, that the benefit of seeing your assumptions crystal clear far outweighs allegiance to traditions that keep those assumptions
8.1.2020 7:19pm - (3/3) cryptic and indefensible. How come? @causalinf , @PHuenermund , @lewbel , @Susan_Athey @pedrohcgs @snavarrol @TymonSloczynski @tdetonberry @MacroPru . The question is interesting because, in other fields (eg. epi) it took only one paper to convince both leaders and masses.

8.1.2020 6:07pm - (Replying to @BenOgorek) The previews give away (essentially) the entire book. The only reason we labeled them "previews" is to keep the publisher from suing us. (Too old to sit in jail).

8.1.2020 5:22pm - I can't praise this book more than I already did but, having just read how economists introduce causality to college students, I am going to praise this book one more time.

8.1.2020 5:14pm - (Replying to @LahavHarkov and @Sethrogen) Moreover, see the facts about what Israelis knew about the Arab population in the land:

8.1.2020 5:11pm - (Replying to @benshapiro and @Sethrogen) Moreover, see He who has been fed lies all his life and swallowed them illiterately is hardly in a position to wake up one day and teach others how to tell lies from falsehood.

8.1.2020 4:57pm - (Replying to @davidharsanyi) Moreover, see

7.31.2020 9:05am - (Replying to @ScienceandLove) I do not see a distinction between "useful intuitive pump" and the way "human think about causality" and, moreover, I do not think it is relevant to ask how causal forces "really" act in the world. As a student of AI, capturing how human so it is already a great achievement.

7.31.2020 5:23am - (1/ ) It occurred to me that Seth Rogen's idiotic remarks about the establishment of Israel ("racist endeavor") are not totally his fault. This chapter of Jewish history has not been part of Jewish education in America. We should correct for it by revisiting
7.31.2020 5:23am - (2/2) (Replying to @yudapearl) and citing and some of those remarkable quotes by Ben Gurion, Weizmann, and Jabotinsky, to set the record straight. @MiriamElman @WarpedMirrorPMB @MattiFriedman @rabbisacks @EinatWilf @HenMazzig @AndrewPessin @ayidindixieland @GilTroy @drdivine @deborahlipstadt @marksjo1)

7.31.2020 4:08am - Eine Kleine Nachtmusic

7.31.2020 3:03am - A must-read for understanding the New Religion and its holy scripture.

7.31.2020 2:30am - (Replying to @mfkuepp and @eliowa) I recommend reading it in English, it rhymes like Mozart.

7.31.2020 1:57am - (Replying to @clovesgtx) Your class notes strike me as an upside down way of teaching causal inference. Why not start with structural equations then show how potential outcomes can be derived organically from those models, as is done here Otherwise, what's the role of graphs?

7.31.2020 1:45am - My readings into the phrase "appropriate study design" made me more cautious of the phrase than of what it intends to exclude.

7.30.2020 1:24am - This favorite lie repeated by Zionophobes, that early Zionists ignored the (equally) indigenous Arab population, prompted me to do some research on the topic. The results are published here: We can forgive Rogen's ignorance, but who were his teachers?

7.29.2020 10:59pm - Rogen is envious of all the media attention Peter Beinart got for saying: "Who needs Israel". Two Jews of discomfort: "Mother, I hate you for make me feel so inadequate. The boys in school called you names, and I couldn't defend you, so I joined them"see

7.29.2020 8:54pm - The greatest lie Rogen tells us is that he was fed lies about absence of Arabs on the land. Ben Gurion's first paper (1917)was a demographic study of those Arabs, his first book: "We and our neighbors (1933)". From kindergarten to college Israelis are told "our future neighbors"

7.29.2020 8:54pm - Seth Rogen: 'I Was Fed a Huge Amount of Lies About Israel My Entire Life' via @jewishjournal

7.29.2020 3:06pm - (Replying to @ThatMarkElliott) I did not realize registration is closed. Will check with the organizers about recording.

7.29.2020 2:03pm - Tomorrow, 9am PST, I will be giving a virtual talk at the CI-FAR workshop on causal inference, My title is almost the same as here But I will spent the 1st 10 minutes discussing Radical Empiricism

7.29.2020 12:22pm - Tomorrow, July 30 is the 9th of Av, the most calamitous day in the Jewish calendar. Please take a look at the golden candelabrum, or Menorah, carved in deep relief, top of this post: I never pass through Rome without paying a visit to Arch of Titus. Why?
7.29.2020 12:22pm - (2/ ) Because it never fails to dazzle me with the profound irony of history: A monument erected to commemorate the eternal end of Jewish independence turning into the most explicit and indisputable testimony of that independence, and the most inspiring symbol of its revival. I ...
7.29.2020 12:22pm - (3/3) I often catch myself wanting to say: Thank you Titus, only to recall that, after all, he was the destroyer of our 2nd Temple.

7.29.2020 11:48am - Kudos to @themuslimreform for joining the @Twitter Walkout. This Wiley gives me an unwanted superiority complex (Jewish privilege?) for having been protected from the cesspools of hate and ignorance in which he must have grown up.

7.29.2020 10:36am - Note that the courts upheld: "the ordinary meaning of 'because of' is 'by reason of' or 'on account of,' ... That term incorporates the but-for causation standard, formally defined using counterfactual logic, as in #Bookofwhy

7.29.2020 10:24am - Glad to be back with readers, having just ended our #48HoursSilence walkout to protest @Twitter role in amplifying the racist voice of Wiley the Conspiracy Maker. True, I usually react by walk-in, not walkout, but, in this case being counted was important.

7.27.2020 10:51am - In solidarity with our readers in UK, Jews and non-Jews, artists, authors and civic leaders, I am joining their call for a 48 hours Twitter Walkout I find it inconceivable that we, scientists, will remain unmoved, distant onlookers.

7.27.2020 10:22am - BBC News - Wiley: Anti-Semitism row prompts 48-hour Twitter boycott

7.27.2020 2:39 (Replying to @yudapearl and @skjask) ... while the bridegroom denies the bride was born.

7.27.2020 2:15am - (Replying to @zacharylipton) Some of them have escaped through graphical models to prepare the grounds for next generation ML:

7.27.2020 2:04am - (Replying to @skjask) Three arm-chair match-makers discussing the menu for the wedding, without asking the bride if she agrees to have one.

7.27.2020 1:47am - The Book of Why has a whole section on it, including the Kruskal-Bikel exchange, reflecting on its lessons. But no discussion of Simpson's paradox is complete without understanding its resolution and its post-resolution longevity:

7.26.2020 7:17pm - The example has a variant with a painfully bitter end, concluding with inevitable regret, and irredeemable remorse. See

7.26.2020 6:19pm - I understand that @smueller taught a causal inference class in high school (using Primer). He may have class notes to share.

7.26.2020 5:42pm - If all knowledge comes from sense data, why don't we just gain knowledge the way our ancestors did, by simulating natural selection on our data? Or the way ML folks do, by fitting functions to data? I tried to answer both questions in a new blog post:

7.26.2020 2:15pm - Rung-3 comprises inferences that require undoing past events, eg, " if only my plane had left on time..." It is explained superbly here:

7.26.2020 5:00am - I am posting this paper "Einstein's Greatest Legacy, Thought Experiments" because thought-experiments are acts of imagination, unique to Rung-3 of the Ladder of Causation, that are a necessary ingredient of any automated scientist.

7.25.2020 5:16pm - A new @JewishJournal story on Carnegie's "Great Immigrants" Award and on how Israeli-American immigrants can save American Jewry:

7.24.2020 10:24am - It's not enough for hundreds, even thousands professors to prove Zionism the most noble and inspiring experiment in human history; the crocodiles will continue to demand their daily meal, until the village elders say: "What an ugly and dangerous animal! No more!"

7.24.2020 9:07am - Thanks, Cyrus, for excavating this ancient 1987 piece from the archives of embryonic ideas. Reminds me of the days when we were truly wondering: Who needs graph? Some people are still wondering, amazing!

7.24.2020 1:35am - (Replying to @rorykoehler) The definition is from the dictionary, but applying it to people with whom you disagree turns you into a bigot, God forbid.

7.24.2020 12:33am - (Replying to @rorykoehler) Please note, your definition of "bigotry" does not express much warmth toward other opinions, say toward those who do find positive sides in certain forms of nationalism. One positive side, eg. is the cultivation of knowledge and experience transmitted across generations.

7.23.2020 12:46pm - (1/2) In solidarity with my friend, Professor Miriam Elman, who was attacked for being "openly Zionist": I would like to openly state that, in addition to being "openly Zionist" I am also openly disgusted with those who are openly Zionophobic and,...
7.23.2020 12:46pm - (2/2) as long as I live, I will try my best to openly expose their bigotry, hypocrisy and moral deformity.

7.23.2020 11:15pm - (Replying to @YablokoU) Zionophobia is treating Zionism as anything less than normative and organic. If you question the fitness of an openly straight person to teach a class on marital psychology then, yes, I will question your motives.

7.23.2020 10:59pm - (Replying to @YablokoU) Zionophobia is a moral deformity in that it denies one people the right to a homeland. Beside, I've never seen a Native American or a Canadian patriot being questioned for fitness to teach history.

7.23.2020 10:40pm - A surprised reader asks: Did you really mean to imply that Potential Outcome folks who do not know graphs do not know potential outcomes? Yes, I meant it. And I have explicated precisely what properties of potential outcome those folks are missing, e.g.,

7.23.2020 4:22pm - (Replying to @aminkarbasi) What permission do you seek?

7.23.2020 3:57pm - I don't know anyone knowledgeable in causal graphical models who is not also familiar with potential outcomes. The latter are derived from the former in 3-steps (see, Section 4.2.4) which tell us what potential outcomes are all about.

7.23.2020 3:04pm - (Replying to @rorykoehler) The positive side of it, yes!

7.23.2020 2:15am - (Replying to @itamarcaspi and @HebAcademy) Behatslacha!

7.23.2020 12:00am - (Replying to @rohan_virani) Hard to comment w/o seeing the transcript. But, off hand, a monkey too learns by playful manipulations of objects yet, unlike a baby, the monkey never grows up to invent a sowing machine. Why? Isn't it because it lacks a template to store its experience? i.e., a causal model.

7.22.2020 1:21pm - A breathtaking discovery in Jerusalem: World-orders are crumbling, new viruses mutating, yet nations are busy unearthing their historical infancy and, at times, the earth is generous -- I have known King Menashe and Hezekiah since 3rd grade (Kings 21:1).

7.22.2020 10:44am - (Replying to @itamarcaspi and @RazRaanan) How about "Bli-Avad", al mishkal "BeDi-Avad". But its more than just "bli", its "Negdi-Avad". My grammar teacher would have flipped, poor Dr. Gotshtein.

7.21.2020 12:53pm - What a beautiful satellite photo of Israel, with patches of lights marking Tel-Aviv, Jerusalem, Haifa, Amman and Cairo. Even Cyprus and Turkey can be seen in the distance, and the Sinai Peninsula amazes you again why it took 40 years for the Israelites to reach the holy land.

7.21.2020 12:19pm - Wonderful post about missingness-graphs, agree! How come we missed it thus far? Does it means missing-data folks are beginning to ask WHY, like the rest of us, and demand to understand what their assumptions mean, and what can be recovered from missing data? End to hopelessness!

7.21.2020 6:05pm - (Replying to @GaryMarcus) Thanks for re-posting, Gary. This came out July 4th, but it's always good to be reminded that I represent an ethnic minority of Israeli-born immigrants who are grateful for the opportunity to share our unique experience and add our distinctive color to the American tapestry.

7.21.2020 2:51pm - (Replying to @itamarcaspi) Eineni Yodea. An interesting question for Vaad Halashon.

7.21.2020 8:18am - How do we know it is a matter of kind, not of degree? Because we can't even write the question in the vocabulary of "what if I do?". Try writing formally: "What is the probability that the dead victim would be alive had the defender not shot". See

7.21.2020 8:08am - (Replying to @BenHanowell @PHuenermund and @eliasbareinboim) Think about causes of effects at the individual level. Was it my aspiring that caused my headache to go away, today, at 3pm, after I watched that movie, and listened to Mozart's serenade? What if my plane had arrived on time, now that my partner is angry and the deal is off?

7.21.2020 7:31am - This paper illuminates the hierarchy that I once drew in the sand with several new lights, more mature and compelling, making me ask again, this time more humbly: How can any data scientist ignore its ramifications?

7.20.2020 10:17am - One thing publishers don't tell you, which is probably more important than the cover or the price: This edition will have all the errata corrected, so you can run through it with no hurdles. Enjoy the run!

7.20.2020 7:48am - [Sorry for the broken tweets.] Its value is explicated here: and elsewhere. However, research questions handled by Tools 3-7 are outside the vocabulary of most "Data Science" folks today, and so are these questions:

7.20.2020 7:41am - Retweeting. I have explicated this value here: 3-7 are currently outside the vocabulary of most Data Science folks, and so are these:

7.20.2020 7:29am - The value of Data Science is obvious to me too, even under its current data-centric direction. I have explicated this value here:

7.20.2020 4:48am - For readers who, like me, grew up in the information age, here is a fascinating public lecture about the life and works of Claude Shannon, the man who brought information age to where it is today: AI folks should also recall his 1948 chess-playing program

7.19.2020 9:07am - (Replying to @willcfleshman) Why would answering questions at the bottom of the ladder be appropriate in your research question is about a higher rung of the ladder?

7.19.2020 8:59am - (Replying to @ShalitUri) The difference between "people w/diabetes" and "entire med. record" seems a matter of degree, whereas "group-level" vs. "individual-level" is a qualitative difference. If so, I am not entirely off claiming (in that the latter requires counterfactual logic

7.19.2020 2:33am - (Replying to @lukasvermeer and @learnfromerror) I would be delighted.

7.19.2020 2:24am - Alchemists had one advantage over data scientists, they could recognize gold if/when they were to produce one. Ask a "data scientist" if her latest neural net answers her scientific question. The answer I often hear: "What question?" This is why I contest the SOUL of Data Science

7.19.2020 1:18am - (Replying to @lukasvermeer and @learnfromerror) Your Goto 2016 talk is brilliant. Is there a transcript or a summary? I would love to taste more of that historical Sauerkraut. It reminds me of a parable I often use: "It is hard to find a needle in a haystack, it is almost impossible if you've never seen one."

7.18.2020 11:19pm - (Replying to @learnfromerror) What data science people aspire to is "Data Intensive Science", which is a lot different from traditional, pre-big-data "science". What they actually practice is a "Science of Summarizing Data" which is hardly a "science". See

7.18.2020 11:01pm - (Replying to @data4sci) I've always questioned the purpose of "working with data". Even when I first heard about "taking the average" I asked "WHY"? Why not go with the last sample? And only when told that the mean gives you a better prediction I was (partially) satisfied. (Proved in 1756).

7.18.2020 5:02pm - I like the name "Data for Science", which counters the current direction of "Data Science", as if Data in themselves are the target of scientific exploration.

7.18.2020 4:05pm - (Replying to @tytung2020) I will look into the alternative explanation of the horrific video that I've shared here. But please do not teach me about Guantanamo - the confessed murderer of my son sits there, still boasting in his accomplishment.

7.18.2020 3:55pm - (Replying to @CasualBrady) It is the same concept, yes. We tried to name the specificity by the characteristics assumed, or the variables which are measured.

7.18.2020 3:51pm - (Replying to @stephensenn) Regression slopes are still population-level properties. There is not "regression slope" for Mr. Jones' reaction to drug D, taken Wednesday 1 pm. Counterfactual logic does tell you something about this singular reaction, see

7.18.2020 3:40pm - A thrilling chapter in history - "20 years ago today": the Camp David Summit. An interesting story from an eye witness, encapsulating the core of the core of the Israeli-Palestinian conflict: MUTUAL ACCEPTANCE of historical legitimacy.

7.18.2020 9:46am - (Replying to @CasualBrady) "c" stands for "characteristics". That where the "c" came from originally (Shpitser and Pearl 200?). But you have a point.

7.18.2020 8:54am - (Replying to @nagpalchirag) Thanks for posting. I like the fact that you do not pose "ignorability" as an apriori axiom, but show how it is derivable from the model of Fig. 2. I recommend that you state explicitly what type of assumptions allow you to estimate effects, given the Z is latent.

7.18.2020 7:11am - Readers noted that my Moment Magazine piece on 1948 is chopped prematurely, and the part where I urge every holocaust museum to dedicate a "From the Ashes" exhibit is hidden away. Here is the hidden part:

7.18.2020 6:41am - Readers ask: How can you claim (in that going from population data to individual level requires counterfactual logic, when ML folks estimate Individual Treatment Effects (ITE) straight from group data (eg Answer: What they
7.18.2020 6:41am - call ITE is not really "individual" but average effects in a specific group that shares the measured characteristics of an individual. We call this average "c-specific effect." Individual-level effects indeed require counterfactual analysis, as is done in

7.18.2020 6:11am - Thanking @eliasbareinboim for sharing his illuminating slides and video on Causal Reinforcement Learning, and his methods of combining the unbiasedness guarantees of Causal Inference with the sampling efficiency of Reinforcement Learning.

7.17.2020 12:51am - Photos of roundups, ordinary folks, in ordinary civilian cloths, carrying ordinary household items, clinging to ordinary logic, are sometimes more horrifying than those of inmates in death camps. Watch the 1942 roundup of Paris Jews, 78 years ago, today:

7.16.2020 10:58pm - I tried to skip this footage, and move on to AI, ML and CI, but my fingers wouldn't let me. They force me to retweet, as wide as I can, because the appointed guardian of these brutalized victims, the UN Human Right Council won't - it's busy condemning Israel for being alive.

7.16.2020 9:19pm - (Replying to @genomixgmailcom) And I can't wait to hear how a hard-core regressionist reacts to my heretical conclusions.

7.16.2020 11:31am - As scientists, it behooves us to do more than point out how absurd the UNHRC has become, and how it stains the reputation UN as a whole. We have psychiatrists and social workers in our ranks, so, don't these delegates and their governments have some sense of right and wrong?

7.15.2020 8:38pm - My favorite beach. That's were I learned to swim, 80 years ago. I've changed somewhat, the waves havn't. I became impatient, the waves didn't. Look at them, endlessly trying to dare the beach into submission - I've given up.

7.15.2020 8:16pm - (Replying to @Claire_Voltaire) In my neighborhood Zionophobes use it more than any other group.

7.15.2020 9:34am - Sharing personal experience from 1948, the year when everything changed, the magnitude of which we, 11-year-olds could not begin to imagine.

7.15.2020 8:40am - (Replying to @michaeldickson @michaelduckson) , when was the last time you used the word "Zionophobia"? The only fighting word left to us, and our leaders and spokespersons are so careful not to use it.

7.15.2020 8:13am - Sharing a fascinating interview with Eran Zaidel - How Brain Scientists Think about Consciousness Closer to Truth, 13 July 2020 Eran Zaidel is a UCLA colleague, another former Israeli who came to this country on an international student visa.

7.15.2020 5:38am - (Replying to @jwbelmon) Funders are rarely evaluated, hence rarely criticized. Funders shield themselves from criticism by re-funding the old guard of PI's, to do more of the same, often under the hype of "new initiatives".

7.15.2020 4:32am - I believe the symbiotic relations between ML and CI is defined by the inference engine of as well as in the more general contest on the soul of data-science:

7.15.2020 4:12am - (Replying to @renatrigiorese) I think the "IN vs. OUT" partitioning can be viewed as compliant with Dennett's intentional stance. It is appealing to me because it is simpler and more operational.

7.15.2020 4:01am - A reader asks: "Some colleagues are insinuating that you subscribe to the probability interpretation of causation from the logic of your do-calculus..." God forbid! The probability interpretation of causation was an unfortunate blunder, discussed here:

7.14.2020 1:13pm - As I read her words: "Twitter has become the ultimate editor," I was thinking, to what extent this is true in our truth-based scientific bubble? To what extent signaling allegiance to a guru, editor or colleagues dictate how we phrase our findings or equations? Still thinking...

7.14.2020 12:54pm - I was shocked this morning to read Bari Weiss' resignation letter (from NYT): I have known Bari since her student's day at Columbia when she bravely stood up for students rights, and she has been on my hero's list since. Freedom has lost a real fighter.

7.14.2020 12:12pm - (Replying to @metadiogenes) The other way around. My issue with entropic formulations is that they DO NOT partition the world, but try to explain agent-asymmetries in physics itself, not in the observer.

7.14.2020 8:20am - (Replying to @metadiogenes) I am not familiar with this formulation, thanks for pointing, but my skepticism to any entropic formulation continues to dim my enthusiasm.

7.14.2020 7:11am - The reason I feel uncomfortable with the answers is that thermodynamic-based features apply equally well to a thermostat, a device we all agree has NO agency. The account I've proposed to explain these asymmetries is based on our habit of partitioning the world into IN vs. OUT
7.14.2020 7:11am - namely, a chunk of the world we provisionally consider the focus of our attention vs. the rest, which we consider "external" to the former (see p. 420 of Epilogue ) This arbitrary choice does not show up in the equations of physics.

7.14.2020 4:19am - Another #JewishPrivelege I cherish: Being bought up in a culture that looks at facts and exercises reason before falling for juvenile conspiratorial theories that Cannon evidently takes very seriously. What a mighty privilege!

7.14.2020 3:33am - This choir of Japan's House of Peace is singing in Hebrew: "When the heart cries". The choir apologizes for having to do it through youtube instead of their usual world-wide tour. "When the heart cries, God does listen".

7.14.2020 3:11am - Apologizing for the occasional mis-syncronization between my voice and my slides which I could not observe in LA, due to a Zoom Dragon acting up over the pacific ocean. But the message is still there: Data-Science should shift its focus from data to science.

7.14.2020 1:09am - (Replying to @MasahikoKim) Given your audience, I believe your team did a great job in jolting the bio-health informatics community to think causally, to leverage ML methods, and to ask themselves which rung of the ladder they wish to climb.

7.13.2020 5:12pm - I always welcome "computational models" in Journals like Congnitive Science but I can't see the "model" when authors refrain from using counterfactual notation to define "sufficient cause" as in or

7.13.2020 6:27am - What is an AGENT? The questions raised in this paper deserve discussion: (i) what makes a physical system an agent; (ii) the reason for agency's time orientation; (iii) the source of the information generated in choosing an action. Unsure about the answers

7.13.2020 1:07am - My personal #JewishPrivilege is that my parents concealed the holocaust from me till I was old enough to stomach it and, this way, ushered me and my generation to grow up into normalcy rather than victim-hood -- what a privilege!!!

7.13.2020 12:26am - I wish I could attend this ICML tutorial today (Monday) on causal reinforcement learning: -- I've always wondered why RL folks don't sit together with CI folks to create a synergistic ML agenda. Webpage:

7.12.2020 2:33am - (Replying to @WiringTheBrain and @DrYohanJohn) If we ask: Why is experience in the world so important for intelligence? How is it encoded in the mind after it is no longer experienced? How it is accessed, etc. we get closer to replacing that experience with a surrogate program, thus rendering that experience unnecessary.

7.11.2020 7:04pm - (Replying to @latinostats) Luckily, human intelligence refused to listen to philosophers speaking and has evolved beautifully in grey organic substance called "brain". Now on its way to experiment with silicons.

7.11.2020 4:48pm - The article is worth reading, I agree. I cannot share however its enthusiasm for, and resurrection of Dreyfus arguments which, IMO, aim to mystify, rather than understand human intelligence. AGI will be achieved by conquering one mini-Turing-test at a time, not by quitting.

7.11.2020 3:48am - (1/2) Now that Beinart has declared himself a "Zionist", thus stripping Jews of the one word that defines their historical identity, let me assure readers that the word Zionophobia still shines in its original clarity: "Denying Jews the right to a homeland of their own." So,
7.11.2020 3:48am - (2/2) So, Beinart may call himself a "Zionist" or a Giraffe, if he wants; the bigotry of Zionophobia will continue to haunt him unambiguously, with all its ugliness.

7.10.2020 11:19am - Jayathi Murphy, the Dean of our school (UCLA, Engineering) has authored a powerful oped on behalf of International Students. I am proud to share it on this educational channel.

7.10.2020 10:20am - (Replying to @daniel_bilar and @ShlomoArgamon) Thanks for clarifying.

7.10.2020 10:19am - (Replying to @Rondo2 and @kweansmom) Beg to differ. Words mean an awful lot, if you teach those words to your children, and reinforce them in schools and mosques.

7.10.2020 9:26am - (Replying to @Rondo2 and @kweansmom) They can easily prove that they are different: Acknowledge some historic Jewish connection to the land. I am waiting.

7.10.2020 8:59am - (Replying to @BlakeFlayton) The term "pro-Israel" is thrown around because we, Israelis and Zionists, have allowed anti-Israelism to become a debatable item, rather than mental disorder. Same with "proud-Zionist"; I've never met a Zionist who was less than proud.

7.10.2020 7:24am - Beinart's first victory = Linda Sarsour! A Zionophobe turned "Zionist".

7.10.2020 7:10am - Any Albanians among our readers? I salute you for your bravery and sacrifice during WW-2! In general, I am not too fond of "holocaust memorials". The best monument to Jewish survival is the rebirth of Israel, which is a living monument that breathes, creates and inspires.

7.10.2020 5:53am (Replying to @devon_l_sch) Happy Friday, Devon. I am familiar with the diagnostic system of Bayesia, and the contact I have is: . I hope it works for you.

7.9.2020 11:43pm - (Replying to @prem_k) Do you believe there is merit to this critics? NonParametric causal inference starts with "if then" specification of what we know (DAGs) and ends with "how much" answers to what we want to know (eg ACE). Gelman missed it, because he refuses to solve a toy problem start-to-end.

7.9.2020 5:02pm - (Replying to @ekirwin8 and @Larry_Svenson) Why get your head spinning in all those so called "frameworks". Just solve ONE toy problem, from beginning to end, and you are ahead of 90% of the spinners.

7.9.2020 3:23pm - (Replying to @kareem_carr) It is easy to agree with me (I am fairly agreeable person) The tough thing is to agree on the mathematics of causation. Do you think your fellow statisticians understand the two fundamental laws of causal inference, or, at the very least, the 3 rungs of the ladder of causation?

7.9.2020 2:11pm - Glad you brought up Gelman's "sort of" review. I always recommend it colleagues who ask about causal thinking among leading statisticians in the 2nd decade of the 21st century. But, I reiterate, one toy example teaches us more about causation than 100 debates among the gurus.

7.9.2020 7:17am - Do We Really Have to Read Beinart Again? via @jewishjournal

7.9.2020 7:17am - Another take I recommend about Beinart's heart-breaking plea for social acceptance.

7.9.2020 6:48am - What are "Jews of discomfort"? What is "indigenous"? Two thoughts pertaining to Peter Beinart's surrender to thoughtlessness.

7.9.2020 5:06am - What part of data analysis is or is not causal is not up to @yudapearl , but comes out of the mathematics of causality. See interview with David Hand on this issue:; tasks believed to require no causal thinking turned out to benefit from it. @kareem_carr

7.8.2020 4:40pm - The famous plea: "Give me Yavne and its scholars" was made when Jerusalem was in flames. @PeterBeinhart distorts it to wet the appetite of Jerusalem's enemies, while she is a thriving center of art & science, and a bastion of moral clarity. Beinart is a catalyst of destruction.

7.8.2020 4:11am - (Replying to @kenanalytics) What's the point? No one doubts the antiquity of Australian natives' culture. This does not negate however the Jewish quest for an "equally indigenous" status, a status denied them by "sole ownership" neighbors.

7.8.2020 2:53am - Another tiny difference: British settlers did not see Australia as the cradle of their civilization, Jews did. Having celebrated homecoming for eighty generations in poetry, prose, lore, holidays, and daily prayers, they saw themselves equally indigenous to the land.

7.8.2020 2:10am - Contesting the Soul of Data-Science. Below, an introduction to the Spanish translation of #Bookofwhy: It expresses my conviction that the data-fitting direction taken by “Data Science” is temporary, soon to be replaced by "Data-intensive Causal Science"

7.8.2020 12:56am - This Foodbenders Lady has taught Toronto a lesson in propaganda. Decent Torontonians should reciprocate: "Some anti-Zionists ARE Welcome in our stores, if they admit they know nothing about the conflict, and haven't realized how genocidal and racist anti-Zionism is."

7.8.2020 12:33am - Foodbenders Owner Backtracks, Now Claims Some Zionists ARE Welcome

7.7.2020 11:38pm - Congratulations from inspired readers remind me that we, ~500,000 Israeli-born Americans, make up an sizable ethnic group with a unique culture and a unique experience to share: Building a new nation (against all odds), now a world center of art, science and entrepreneurship.

7.7.2020 1:09pm - As promised, the Spanish translation of #Bookofwhy is finally out It gives me a tremendous pleasure to think that my playful thoughts will be read in the language of Cervantes, Goya and Picasso. So, read it slowly, and with a spec of forgiveness.

7.7.2020 11:00am - As one of the few survivors who participated in its inauguration (UCLA, Aug. 1985) I am proud to announce this year's Conference on Uncertainty in Artificial Intelligence (UAI) to be held FULLY VIRTUAL -- Aug 3-6, 2020. Details here

7.6.2020 9:51pm - "Race, COVID Mortality, and Simpson's Paradox." Dana Mackenzie, my co-author in #Bookofwhy, has discovered another case of Simpson's Paradox in COVID data, and has posted an interesting analysis of its implications here:

7.6.2020 12:34am - These are important papers that should be read by all ML folks who talk about "transfer learning," "domain adaptation," or "lifelong learning". I wonder if the Epi folks are aware of these general methods for "counterfactual predictions" @barbradickerman @_MiguelHernan

7.5.2020 11:29pm - The problem of external validity has many variants, depending of what data is available from each setting and how the settings differ from each other. Once we encode this information in a selection diagram, the solution pops up algorithmically, as in

7.5.2020 9:01pm - The problem of deploying one prediction model in a new settings seems to lie within the umbrella of problems called "external validity". Could selection diagrams be of use here, to denote the disparity between the two settings? eg. as in

7.5.2020 2:55am - A great post with which we can reflect perhaps on our educational and research institutions and how they manage to keep academics toeing the party line.

7.5.2020 1:22am - More on Simpson's paradox (SP). As Covid-19 stimulates greater public interest in data analysis, it also makes more people wonder about SP and its implications. Unfortunately, most published accounts of SP are flawed, for they skip its causal roots and causal logic, 1/3
see: Readers who wish to get straight to the point, should take a look at Fig. 6.6 (p. 212) of #Bookofwhy, and ask themselves two questions: (1) Why are you surprised? (2) What is the correct action? You will get a better understanding from this quick 2/3 7.5.2020 1:22am - see: Readers who wish to get straight to the point, should take a look at Fig. 6.6 (p. 212) of #Bookofwhy, and ask themselves two questions: (1) Why are you surprised? (2) What is the correct action? You will get a better understanding from this quick 2/3
7.5.2020 1:22am - exercise than from most explanations I can find in the literature. Same figure can also be seen here: (fig. 1.1 1.2, p.3-4), and requires no numerical tables, and no arithmetical operations to get to the main point. It is thus a "Critique of Pure Reason."

7.4.2020 11:38pm - I am echoing the beautiful words of Sarai, Miss Iraq, with whom I had the fortune of singing duet last year, who grew up in the same neighborhood as my wife, and who speaks for all freedom-seeking Muslim women, betrayed by their political "leaders" and hate-peddling activists.

7.4.2020 2:29am - RL vs. CBN. In Causality p. 24 a Causal Bayesian Network (CBN) is defined as a parsimonious representation of all possible interventions. This paper seems to reason in the space of interventions, not their CBN. Transparency is lost; what is gained?

7.4.2020 2:02am - (Replying to @EricTopol @CarnegieCorp and 2 others) Strange, when people label me an "immigrant", I feel mighty privileged, not as a member of an under-privileged under-represented minority. This is perhaps unique to US and Israel; two countries were differences are virtue. Happy 4th of July.

7.3.2020 12:14pm - This wikipedia entry on Savage's principle is OK, except that Jeffrey could not say "unaffected by the action". He did not have causal language and had to say "unrelated to the action" which is an overkill. See

7.3.2020 11:01am - (Replying to @AnnaPodolanczuk) B'Yeshimon U'Baarava we called this book in Hebrew. Then there were Pan Volodyovsky, Quo Vadis, The Deludge, By Fire and Sword. Wow, this guy really shaped our imagination. I ask my grandchildren if they read Scienkiewizc and I hardly get: "What?" Immigrants have more in common.

7.3.2020 10:37am - Great post! This is indeed a quote from the #Bookofwhy (p. 87) paraphrased and attributed to S. Karlin. Note, isn't it precisely what we are hearing from researchers locked into Big Data, ML, Deep Learning etc. "The answers are all in the data, including culture and evolution."

7.3.2020 6:46am - Video clips like this one make you pause and think, how many moments in history you've almost forgot and how many heroes of history you've barely ever known

7.3.2020 3:29am - My old Tweet may be relevant here: : "Who needs causality if all you want is prediction?". Ask any pollster what the secret is to good prediction and the answer will soaked in causal vocabulary. Why? Because integrating data from multiple sources (Fusion) is a causal exercise.

7.3.2020 3:02am - What a small world! Your picture is next to mine: and my mother is from Kielce, Poland, just 200km from your hometown, Kolonowskie. Moreover, I grew up with the books of Henryk Sienkiewicz on my lips, did you? Small world!

7.3.2020 2:35am - A slight correction. More dangerous than the untestability of the assumptions is their non-comprehensibility; e.g., the economist does not know if they are defensible or not, even given an unassailable economic theory. Try it on your economist colleague next door.

7.2.2020 11:56pm - My, My! And this was 15 years before Marilyn vos Savant's posed it in her column. So it was your father who first recognized that the problem can be turned into an intriguing guessing game. Please convey my admiration.

7.2.2020 12:07pm - Good news indeed. Easier sleep too. And thank you @AsraNomani for extracting this valuable information from the lawyers and the Press. May September bring us #justiceForDanielPearl

7.2.2020 9:36am - I could not agree more, this visualization is great for teaching. Plus, do not miss the sequel where it is applied to Covid-19 questions such as "patients in greater need"

7.2.2020 3:39am - (Replying to @riemannzeta) I tried to resonate to some of his ideas, but could not (though I liked his use of "Market Blanket", with which I am familiar). I think we are heading in opposite directions, Friston trying to philosophize computation while I'm trying to algorithmitize philosophy.

7.2.2020 2:06am - I have been hearing those same tired slogans since 1936, so where is the "rage"?

7.1.2020 9:09pm - Proud and privileged to be part of this creative group of immigrants, painting together the most colorful mural of the American journey.

7.1.2020 3:54am - (Replying to @s_kampakis) Amazing, this is the first Tweeted post with which I had no disagreement. Am I getting old? Wise? Forgiving?

7.1.2020 3:50am - In causal diagrams EVERY variable is understood to be a moderator, no need for special treatment. On the contrary, special consideration is given to NON-moderators, as in linear models. Can we tell them apart from data? See a beautiful, under-rated paper.

6.30.2020 11:09pm - (Replying to @neuro_data @RaiaHadsell and 5 others) Your manuscript promises to teach me all about "transfer learning" and why I was't able to do it from other manuscripts. My novice attempt to formulate the problem in statistical terms ended up with many open questions: Is there a non-empty interesection?

6.30.2020 2:49pm - (Replying to @desai_pratik) Whatever injustice is perceived by one side or another can be undone with one sentence: "Both peoples have the equal right to be masters of their fate in a state of their own in this land - a Jewish state, Israel, and an Arab state, Palestine.” Time for a new dawn.

6.30.2020 2:42pm - I did not know this 2013 picture with Carl Reiner existed. I will cherish it dearly. Thanks, and condolences to his family and to his many many admirers.

6.30.2020 6:18am - To our Irish readers: May you be served well by your new Government! May it be a new dawn! A new age of reason. Amen!

6.30.2020 6:08am - Tough decision for a leader: Getting his people a state of their own, on the one hand, and admitting that the past 72 years of denying Israel were a futile and unnecessary suffering, on the other. Tough decision.

6.29.2020 5:41am - Thanks you @AsraNomani and all readers who have been staying vigilant with us awaiting #justicefordanielpearl. May reason prevail in September.

6.29.2020 1:55am - I wish it was so easy but, today, saying: "I am against discrimination" automatically makes you an enemy of some people. E.g., The California legislature just voted to strike down the words "not discriminate" from its constitution. Orwellianism: You can't signal loyalty to anyone

6.29.2020 1:18am - Suppose it won't, then what? What if anti-Zionists come to truly love Israel-hating Jews like Neturey Kartha or Noam Chomsky, would that make them less ugly? Less dangerous? Would that offer identity-Jews any comfort? Would it make them more fit to fight injustice anywhere?

6.28.2020 11:53pm - Reiterating: Why does it matter if Zionophobia IS antisemitism or just another form of racism? The question is how genocidal it is in its aims and how deceptive it is in its tactics. To be effective, we must understand and target the unique characters of those aims and tactics.

6.28.2020 11:19pm - (Replying to @HenMazzig) This is just ain't true and insisting on this equation is weakening the fight against the more dangerous form of hate: Zionophobia. It's time that we quit the anti-Semitic debate and focus on the more pathological and more genocidal threat against the Jewish people - Zionophobia.

6.28.2020 5:00pm - It is not what one fails to consider that counts but the consequences of that failure. In the case of PO, what's eroded is the credibility of the conclusions; they might as well run regression. But PO sells itself as a framework for causal analysis, not regression analysis.

6.28.2020 6:43am - (Replying to @BachmannRudi @PHuenermund and 4 others) Sure. My next tweet explicates it:

6.28.2020 4:41am - I have my own theory on what happened to economics and why it has not escaped the PO darkness, but I was hoping to hear first from insiders who have had a chance to solve a problem or two both ways and compare. EG. @pedrohcgs @PHuenermund @causalinf @snavarrol @TymonSloczynski

6.27.2020 10:47pm - (Replying to @dlmillimet @pedrohcgs and 2 others) I have tried to learn from your sausage, but could not tell if you meant "how econ. sausage IS made" vs. "how econ. sausage SHOULD BE made". I am eager to see the "IS" and "SHOULD" applied to the toy example cited here:, so we can see every step.

6.27.2020 8:26pm - (Replying to @EnzoInman) Primer gets you straight into problem solving, using stories and examples (eg Causality spends more time on proofs and philosophy. The do-calculus, for example, is not used in Primer, nor do we compare DAGs with PO. But we you get curious to continue.

6.27.2020 1:48pm - (Replying to @Prof_Livengood) Statistics lack of uptake is easy to explain, given the purging of causality by two powerful founders, Pearson and Fisher: The real puzzle is economics, which pioneered some foundations of causality in the 1940-1970, then invaded by foreign ideologies...

6.27.2020 12:27pm - (Replying to @19Naranjito82) I cannot buy the "there is no right and wrong" argument, unless it comes from someone who tries the X->Y->Z example in and concludes: "no right and wrong". I have not met such a person yet. Economists avoid toy examples because they show the right & wrong.

6.27.2020 9:33am - I am waiting for someone to ask: How is it possible that a whole field of extremely smart people will remain in darkness for 2 decades, when flash lights are readily available? It is really hard to believe, but several explanations have been offered, none entirely satisfactory.

6.27.2020 8:59am - (Replying to @PHuenermund and @krisgulati) Agree! and highly recommended! Is it published/submitted now? Or waiting for an enlightened editor that recognizes merit?

6.27.2020 7:17am - (Replying to @krisgulati) Wow, you are in trouble! PO in 2020? Actually, you are already way ahead of your class (and your instructor). Just learn the translation to graphs and, whenever your professor blushes or hand waves, just help him/her out. Don't reveal how you did it, they won't appreciate it.

6.27.2020 7:09am - I have an example that can clarify my last Tweet day & night. Consider a simple causal chain X--->Y--->Z. Try to express it in PO notation, and answer some questions about it. It is done here, Eqs 6-8. Try it yourself, step by step, and see the darkness.

6.27.2020 6:10am - (Replying to @craigwpickett) If you want to learn to solve concrete problems start with Primer. If you want to philosophize and argue about who did what, when, how, and why not earlier, start with #Bookofwhy

6.27.2020 4:41am - A fair question, that comes up again and again. The darkness behind PO is the requirement to encode knowledge (read: assumptions) in a form that is cognitively formidable to the knowledge provider. See A toy example will speak louder than I can.

6.27.2020 1:55am - I will never get tired recommending Primer, while browsing through #Bookofwhy for fun. But be prepared for a traumatic paradigm shift upon learning that "causal statistics" is an oxymoron, like a "squared circle".

6.27.2020 1:28am - (1/2) Readers will enjoy seeing what happens when economists, bent on avoiding graphs, try to tackle data-fusion problems using PO vocabulary. The result is laid out here: and confirms what #Bookofwhy (p.283) describes as a collection of ignorability
6.27.2020 1:28am - (2/2) assumptions crafted to "justify available statistical methods" with no one left to tell when/if they hold or not. Compare to the way data-fusion problems are solved using graphs: and . Is light just another face of darkness?

6.26.2020 10:51pm - I am glad to see DAG's and colliders leveraged to clarify "systemic bias" and other hotly debated notions. Note however that the checker-board display is, again, statistical, thus concealing the causal forces behind the proportions. The cited paper by Knox etal shows both.

6.26.2020 10:36pm - A nation lacking nationalism is what we call "benign nationalism" which is, indeed, what makes some nations "great".

6.26.2020 2:07pm - This is the most close-up video of a car ramming scenario I've seen in my life. The body you see flying out of the kiosk is that of a female soldier, who suffered only minor injuries. The driver, through the Palestinian lens of truth, is on his way to a wedding. No words!

6.25.2020 11:05pm - The only way to curb this growing cult of inhumanity is to disseminate the video as broadly as possible; Shame-striken Arabic-speaking viewers will then realize what's being done to the reputation of their culture and heritage, and will break their silence.

6.25.2020 9:11pm - (Replying to @parijat_267 and @daniel_serman) That's an interesting take which, as a scientist, I would entertain only after trying my best to refute, hoping to capture formally at least most features of "fairness" that we sapiens all share. Recall, the "no universal" excuse was tried on "beliefs", then on "causes".

6.25.2020 3:59pm - (Replying to @ehudkar) Phrased this way, it makes perfect sense, and I would be the last one to label it "trivial".

6.25.2020 2:33pm - If there is one thing the ladder of causation tells us, it is that it is impossible to get "causal influence" from data + logistic regression or ANY kind of regression. The latter is rung-1 while the former is rung-2. So I cannot parse the question.

6.25.2020 8:40am - Why? I think this piece created a wonderful opportunity to show people what DAGs can actually do, and how misguided and disoriented their critics are. See my take on the Tale and on pluralism:

6.25.2020 8:23am - @arnoldroth is my good friend, and his daughter Malki is still my angel. This stomach-turning video (below) of her smiley murderer, boasting on Jordanian TV, should be played in every life-asserting march & rally, to the tune of: Malki Life Matters.

6.25.2020 1:21am - (Replying to @19Naranjito82) A criterion that sits behind a door sounds interesting already. Where can I hear more about it?

6.25.2020 1:04am - (Replying to @Faldict and @daniel_serman) The theoretical foundation for fairness analysis rests on mediation and counterfactual logic, eg.#Bookofwhy Ch. 9, or Recent papers building on this foundation are: and

6.23.2020 6:59am - Today, June 23, marks 25 years since the death Jonas Salk, the man who eradicated polio, and whose quote: “The most important question we must ask ourselves is, 'Are we being good ancestors?'” reminds me again why we erect statures in our town squares.

6.23.2020 6:33am - (Replying to @edsonedge) Allegiance to cultural taboos is one reason, others are described here:

6.23.2020 6:17am - Who would imagine that the Journal of Applied Economics could publish a paper using Causal Bayesian Networks to analyze food security: The authors are from China, where they have not been lectured yet on why graphical models are not useful in economics.

6.23.2020 2:29am - As a grad student who landed in NYC in 1960 with $56 in his pocket on a special "scientist quota," I promise all stranded grad students that we, American faculty, will do everything in our power, however limited, to help you circumvent the new draconian restrictions.

6.23.2020 1:29am - (Replying to @BlakeFlayton) Well put. And let's not forget that this sort of rhetoric is cultivated by many Jewish leaders, who would scream from the roof tops upon hearing "Benjamins" or "controlling the media" but would cut deals with their ideological comrades who just want to do away with Israel.

6.22.2020 9:40pm - (Replying to @primedivider I am always open to new ideas, and my email is public.

6.22.2020 9:36pm - (Replying to @analisereal and @f2harrell) I had the same difficulty understanding what the research question is/was.

6.22.2020 6:07am - (Replying to @victorstorchan) If I may, what is "predictive justice"?

6.22.2020 4:50am - (Replying to @garethrobertsza) I tried to respect egos and prides. But, frankly, examining the past decade, can you spot a conceptual breakthrough, or an out-of-the-boxish insight?

6.22.2020 4:37am - Great discussion! Agree! Thanks for posting.

6.22.2020 3:23am - For readers waiting for an insightful article by an economist, this one is the best I have seen in a decade: It offers a platform for modeling an agent's state of belief which is different than yours, analyzing and rectifying causal misrepresentations.

6.22.2020 2:22am - Congratulations to our esteemed colleague Daphne Koller who has just won the 2019 Allen Newell Prize,computer%20science%20and%20other%20disciplines. Daphne is the co-author of the book "Probabilistic Graphical Models" and the co-founder of Coursera Congratulations!!! Daphne.

6.21.2020 6:34am - (Replying to @DrBobGoldberg) Of course not. They only intimidate the ignorant and the gullible, not folks who may expose what they really stand for.

6.21.2020 4:04am - Paying my dues to history, and to collective memory.

6.21.2020 3:45am - (Replying to @ToddTheLinguist) By "leaders" I mean people like SF Mayor who was afraid to ask: "Are you crazy?", and only commented on "the cost of re-painting".There are also academic leaders, who rush to take down whatever can be stained with "racism", however remotely.

6.21.2020 2:47am - (Replying to @zhimin_xiao) Perhaps because I still have a dual American-Israeli citizenship, or because I have a visiting position at the Technion, Haifa. Or, perhaps they wanted to make fun of the BDS movement -- I hope it's the latter.

6.21.2020 1:45am - Just finished my Zoom talk at and my host says 165,000 attended, almost as many as stood with me at the foot of Mt. Sinai (remember?). Could China surpass the US in Causal Inference? They surely have a much smaller investment in data-fitting.

6.20.2020 11:56pm - (1/ ) I am always amazed at the depth of wisdom this verse radiates: "Thou shall not oppress the stranger, because you were once a stranger in the land of Egypt" (Exodus 22:21). Meaning: collective memory is the basis of ethics and values, personal experience is not rich enough. 1/2
6.20.2020 11:56pm - (2/ ) Corollary: Common memories inspire unity, unity then inspires willingness to share values; you can't have shared values among total strangers, because "value" is a distillation of accumulated experience. In summary, the DAG is: Memory--->Unity--->Value

6.20.2020 1:59pm - (Replying to @diydatascience @BHilbush and @tdietterich) True. It's just "one or two examples":, "they are only toppling statues", "its the cost of re-painting" (SF Mayor), and "It's just one Reinoceros!" (Ionesco,1959) I hope it ends here, but it's our duty to say: "it may end THERE."

6.20.2020 5:29am - With "Aunt Jemima", "Columbus Day", and even "Fisher Lecture" gone with the wind, here is a panoramic view, from Jerusalem, of America's new memory-smashing psychosis: Where will it end? A nation lacking memory is a nation lacking unity.

6.20.2020 4:34am - (1/4) Comments on your Front-Door paper: * The expression "a single, strictly exogenous mediator variable" is problematic: (1) Causality p. 82 defines FDC as "A set of variables", not "a single variable". (2) "exogenous mediator" is an oxymoron. I originally called it (1973):
6.20.2020 4:34am - (2/ ) "Mediating Instrumental Variables", best described as an "exogenously-disturbed mediator". * "The first application of FDC" sounds too pessimistic. Situations involving exogenously-disturbed mediators are at least as plausible as "exclusion-restricted
6.20.2020 4:34am - (3/ ) exogenous variables" (traditional IV's) which were introduced 70 yrs earlier, when DAGs were not around to invite scrutiny. Imbens comments reflect that absence * Why introduce FDC in the context of linear regression where ATE is identifiable by
6.20.2020 4:34am - (4/ ) Wright's rule The miracle of FDC stems from its non-parametric identifiability. * The do(X) operator was not introduced as a shorthand for "a variable that is randomly assigned", far from it, see Actions need no randomization.

6.20.2020 2:49am - Reflecting on @DerekHelbing's, "crowd wisdom" begins with scouts that travel independent paths. "Crowd madness" sweeps when scouts read the same newspaper and are afraid to report what they read elsewhere.

6.19.2020 4:15am - Speaking about fact and fiction, data and theory, science and opinion, this hardest of facts should turn every Israel-basher into a speechless pillar of salt, and every BDS rhetoric into a pitiful joke. ONE FULL DAY OF PEACE.

6.19.2020 3:41am - (Replying to @yudapearl and @JKugelgen) I believe your paper will be clearer if it starts with a formal expression of your target quantity, to convince us that it resides on Rung-3, not Rung-2, thus requiring SCM specification. Your second approach appears to yield a Rung-2 quantity, which leaves readers perplexed.

6.19.2020 2:18am - Good catch. It should have read: "the guillotine rule of 1793".

6.19.2020 2:15am - Would love to go back 5 years, undo my sins, and renew our dance in the socially-tight #epilove and #epitwitter family.

6.19.2020 2:06am - And we, data scientists, used to believe in the objectivity of data or, at the very least, in the protective shield of mathematical truths. No more! But who is to blame for the guillotine rule of 1973? The Jacobins? Or those who did not scream "Are you out of your mind?"

6.19.2020 1:27am - President Trump and @MaxBlumenthal !!! What a heavenly match!! One consumed by ego, the other by Israel-bashing convulsions. Read on Blumenthal and Elie Wiesel. Endangered species, so it seems, can attract each other from unimaginable distances.

6.19.2020 12:08am - His memory is a blessing. I will never forget his courageous stance against some pseudo-progressives who chastised him for being a staunch Zionist; he won't budge an inch. "Look at yourselves in the mirror" he said (paraphrased) "Is your humanity dead when it comes to Israel?"

6.18.2020 11:39pm - (Replying to @JKugelgen) Question: If we have the causal graph, we can get the causal effects on the actual group seeking recourse and, then, the best we can do is reach Tian's bounds (combining observational & experimental studies), as in Are you saying we can do better?

6.18.2020 10:44pm - A colleague was gracious enough to send me a picture of the Chinese translation of #Bookofwhy, Let's welcome our Chinese readers into the movement of redirecting data-science towards its scientific aims.

6.18.2020 12:52am - Glad the ML community is aware of the need of Causal Science. The next step is to train 20% of their PhD's to write down a formula for the sentence: "It wasn't the aspirin, but the good news that cured my headache" and 10% to model it.

6.18.2020 12:12am - (Replying to @sai_prasanna) Yes, this is what we mean by: opening up a M shaped path, the path X <- A - C -> Y does not exist in the graph, so we prefer not use this terminology, though some European writers do imagine such paths.

6.16.2020 11:39pm - (Replying to @resourcefulco and @analisereal) Good catch. Embarrassment saved. Happy sailing.

6.16.2020 10:30pm - Many thanks to @analisreal for sharing the 3-line solution to the new Napkin Problem (#Bookofwhy p.240), which explains why the baby was so happy, and why I am so confident that, in 3-5 years, every PhD in Epi, Stat, ML, or Econ. will be able to do it.

6.16.2020 7:28pm - (Replying to @analyticsinme @geoffreyhinton and 11 others) I doubt I would make it to this stellar list had they known my resistance to the data-centric direction "data-science" is taking. eg.

6.16.2020 1:04am - Extremely gratified to see the Change petition exceeding 10,000 signatures I hope the Pakistani Supreme Court takes notice and reverses the ruling to free the murderers of my son, Daniel.

6.16.2020 12:31am - Why is he so happy? Good question! But put yourself in his shoes: When was the last time you've found a three-line proof to a problem that would take the average PhD in your generation three months of hard labor? Note, this is the new Napkin Problem, #Bookofwhy page 240.

6.16.2020 12:05am - I am wondering, Ron, Were there other compromises suggested? The Fisher Lecture, from what I understand, has become an Institute all by itself, more stoney than a statue. Were there other ideas to save the Institute and stone the man?

6.15.2020 5:11am - (Replying to @AndersHuitfeldt @ESteyerberg and 3 others) From what I know about non-parametric causal models, they are all about effect measures, in the sense that, combined with data, they allow you to compute any effect measure you wish. Collapsability, however, is purely statistical notion, defined by conditioning, not by do(x).

6.15.2020 5:05am - (Replying to @ESteyerberg @tmorris_mrc and 2 others) Agree. Collapsability is a statistical notion, see (insensitivity to conditioning), and has little to do with "causality" or "effects".(see Causality p.193). A tenuous connection emerges under the assumption of "stability" (p. 194)

6.15.2020 12:04am - (Replying to @rodakker and @JudaPearl) And I wholeheartedly agree with your focus on the hypothetical nature of modern science. When folks object: "but I am not sure about the diagram" I answer: What if you were sure, would you know how to proceed? or what to test? Or how sensitive the answer is to what you dont know?

6.14.2020 11:54pm - (Replying to @Ron_Wasserstein and @AmstatNews) A suggested compromise: Keep the lecture name and change the honor. Replace "in recognition of the English statistician and biologist Ronald Fisher," to read: "in recognition of the scientific achievements of the English statistician and biologist Ronald Fisher." @daniela_witten

6.14.2020 10:10pm - (Replying to @Vitaly84938575) Publishers are notorious for deliberately hiding this information. One way is to look up our Errata Sheet and check if the corrections were done.

6.14.2020 7:28pm - Good news to all readers who have been waiting patiently for the paperback edition of #Bookofwhy. Our publisher says: and What the publisher hides is that this edition will CORRECT all errors that you have helped us detect. Enjoy!

6.14.2020 4:32pm - (Replying to @kareem_carr and @jodiecongirl) I would forgive the irony if it was only in usage of words, not usage of tools. But I think you are right that stat folks emphasize analysis of variance more than ML folks. Both are stuck, of course, at Rung-1, borrowing terminology from higher rungs, eg "explain" "attribute".

6.14.2020 3:19pm - (Replying to @kareem_carr and @jodiecongirl) What is so important about the "can shrink to zero" criterion that deserves a distinction between such two heavy words like "inference" and "prediction"? Will guessing the age of a skeleton amount to "prediction" just because we will never know it for sure? Hard to buy?

6.14.2020 3:04pm - (Replying to @TuanAIWS) Interesting. Has anyone summarized the causal rules of Buddhism in a Tweetable format?

6.14.2020 6:46am - (Replying to @jodiecongirl and @kareem_carr) I think by "inference" you mean "estimating conditional expectation", which embraces both prediction and redrodiction. Would guessing your parents income be "prediction" or "inference"? As we can tell from our Tweets, statisticians are still choosing their terms.

6.14.2020 5:52am - (Replying to @sometimes_data and @kareem_carr) Thanks for the pointer. Note that Efron does not use the word "inference" and that he borrows the causal notion of "attribution" to mean "statistical significance" (variance reduction), confirming my observation that statistics has remained causality-blind

6.14.2020 1:50am - I've found this new article to provide the most thoughtful narrative of the destruction of American news media and, by implication, the destruction of academia as well. I'm still optimistic, knowing that people with no salary to protect WILL speak up.

6.13.2020 11:58pm - (Replying to @Abhishe45860449) My colleagues in ML proud themselves on using NO MODEL. So, I do not know how to answer your question. The role of causality? To do better than: User: Why was my loan was denied? Explainable ML: Because we used Dirichlet prior

6.13.2020 11:47pm - (Replying to @jodiecongirl and @kareem_carr) Thanks for elaborating. But this is the first I hear that statisticians care about "effects" and "interpretability" . I thought these notions are not part of statistics proper. E.g. is estimating likelihood of a disease, given a set of symptoms a "prediction" or "inference" task?

6.13.2020 10:53pm - Counterfactual reasoning in the age of normalized tantrum.

6.13.2020 2:01pm - That's a good point. If we wish to compare natural talents, we should use the "per literate capita" metric. But to compare the ability and commitment of society to investment in education, the "per capita" metric would be more appropriate.

6.13.2020 3:10am - This is the first time I see the front-door formula estimated using Inverse Probability Weighing. Great to hear the wheels of progress squeaking.

6.13.2020 2:38am - (1/2) I did not come from academic family. My father toiled in the orange groves. But my teachers were giants, as I describe here (p.7) and here Oh, and one should not forget my home town, Bnai Brak, founded in 1924, which was named
6.13.2020 2:38am - (2/2) after the Mishnaic town which, in 2nd century AD became a Center of High Learning. Far fetched? Not to us, kids; we knew that those 2nd century sages were watching us daily and expecting us to follow the tradition of deep learning. No excuse.

6.13.2020 1:47am - (Replying to @rodakker) An ideal randomized experiment is still "ideal", and the uncertainties in practical RCT's are not truly "known", for they involve selection bias and other imperfections that are assumed small using "causal assumptions"; same assumptions that RCT's are supposed to escape.

6.13.2020 1:23am - (Replying to @EylonALevy and @EinatWilf) Interesting. Many Indians feel that Winston's status should go down because he was "directly responsible" for famine death of 3 million Indians. Luther was only "indirectly responsible" for European antisemitism and its resulting murder of ??? million Jews. Direct vs Indirect

6.13.2020 1:09am - (Replying to @EinatWilf) I can't resist the temptation of asking (hypothetically) how many of those bemoaning "destroy the two-state solution" would switch position had it been the case that a given act would actually "advance the two-state solution"

6.13.2020 12:29am - Beg to differ. You don't turn an apple into an orange by looking at it through colored eyeglasses. #Bookofwhy (pp. 143-150) shows why randomized trials are merely skillful interrogations of effects that exist with or w/o the trial. An apple is an apple when properly defined.

6.12.2020 11:54pm - My My!! He got it right!! The most inspiring photo I've seen on Twitter in the past year!!

6.12.2020 11:30pm - This is not merely good, it is the GREATER good. What we now need is an authorized list of people/movements to question, or mildly criticize, or less than deify; the hate list is way outdated.

6.12.2020 11:07pm - (Replying to @TuanAIWS) Wait a minute, @TuanAIWS , streets are named after dead people, and I still have 36 productive years left to explore the truly hard questions.

6.12.2020 10:52pm - OK, I confess, the joy is mixed with pride of seeing Israel rank 1st in # of Turing Laureates (per capita), where "pride" is given CI semantics, i.e., an assignment credit to a culture that encourages deep learning and independent thinking.

6.12.2020 10:32pm - (Replying to @BoredAlanTuring) You look awfully familiar! #honestly

6.12.2020 4:43pm - What a joy to see a street named after Alan Turing spelled in Hebrew and English, in Herziliya (named after Theodor Herzl), 5 miles north of my home town. May we live to be worthy of these legacies.

6.12.2020 1:03am - Your question (paraphrased): "Why not let model-free machine learning simulate evolution and reach human-level intelligence as our ancestors did?" was considered by Turing (1950). I touch on it here (~4 min) using the snake and eagle analogy.

6.11.2020 4:34am - This makes me admire the Hebrew Bible. King David is arguably the most admired of all its ancient heroes and, yet, the authors do not attempt to whitewash any of his weaknesses, including murders and moral atrocities that transcend time-sensitive norms. Time for every season.

6.11.2020 3:38am - (Replying to @pratikmehta94) This may be asking too much, I realize. At the same time you can't deny the British people, and their allies, the memory of those days in 1940-1 when they stood alone and prevailed. And (as I tweeted before) collective memory requires a leader's face to be encoded and transmitted

6.11.2020 3:19am - (Replying to @adabhishekdabas) DAG's bless us with a logic that protects us from fallacies such as "good for men, good for women, bad for people". This protection transcends data, bias and sample size. Dont you feel safer being protected from conclusions that our mind finds offensive?

6.10.2020 11:43pm - (Replying to @maliniw90th) When all the statues are down, the town squares are empty, and children books too, just dragons and super-women -- no historical figures, it is then when I will open my first science book, to play chess w/ Pythagoras and Archimedes, admiring their victories, forgiving their sins.

6.10.2020 11:21pm - (Replying to @likeavass) Please re-read my tweet. The words "allegation of blemishes" was used for Gandhi's critics. I hope I have not lost you after all.

6.10.2020 10:40pm - (1/ ) My stand on Churchill and his statue acknowledges the dark side(s) of the man, and my underestimating the intensity to which he is hated by people who were victims of his policies, from Indians to Poles. Still, statues are erected to glorify ideas, and hours of greatness in
6.10.2020 10:40pm - (2/ ) a nation history. Defacing a statue mocks a nation's hours of honor and, indirectly, mocks the lessons we wish our grandchildren to inherit from our experience. And, yes, we must continue to embody those lessons in a face of a person, not abstract symbols, because our mind's
6.10.2020 10:40pm - (3/ ) eye operates thus; we say "Einstein", instead of "relativity"; we say "Galileo" instead of "experimental methods". Faces make us feel part of the idea. So let us feel free to admire Thomas Jefferson for what he means to us, in front of an un-defaced Jefferson Memorial,
6.10.2020 10:40pm - (4/4) acknowledging his ownership of slaves. And let us admire Mahatma Gandhi for what he means to us, precisely as he is depicted in his monument in Delhi, and leave allegations of blemishes to scholarly articles. Our grandchildren beg us for clarity of ideas, cast in human face.

6.10.2020 1:29am - (Replying to @AvanteQuora) "Graphical" models provide the vocabulary for CI. But "probabilistic" models, by definition, remain in Rung-1, unless the instructor/author is enlightened and is aware of the opportunities to climb up.

6.10.2020 12:59am - (Replying to @Abhishe45860449) Mathematics tells us the we dont get a unique solution if we have 3 equations and 4 unknown. This is precisely what the mathematics of CI tells us: For every causal question, data alone are always compatible with both an answer and its negation. See Simpsn

6.10.2020 12:37am - The BDS clowns will stop their campus circus ONLY when each of their so-called "resolutions" ends up with the billion dollar question, and their true face is exposed to the entire campus. They feed on the ignorant and gullible, and cannot survive the light of truth.

6.10.2020 12:25am - Good question, which every ML researcher asks. The answer is 2-fold: (1) Mathematics tells us we can't go above Rung-1 w/o some causal assumptions. (2) To grasp how mild the assumptions, and what we can do once we make them, we must study the logic of CI, not statistics.

6.9.2020 11:59pm - UC Davis Student President Vetoes BDS Resolution And listen to the perpetual billion dollar question: "I can commit to supporting a Jewish and democratic state of Israel peacefully coexisting with it's Palestinian neighbor. Can you do the same?" - Nada!

6.9.2020 3:29am - (Replying to @raymondshpeley and @krysdolega) I was not aware of this side of Churchill, thanks for chiming in. Still, the statue defaced was not erected for the man as much as it was for his great hour. We need more such statues in our public squares for there aren't many great hours left to commemorate.

6.9.2020 12:54am - The defaced status of Winston Churchill is a microscope to the mentality and intellectual making of the BDS movement and its leaders. History will forever contrast the greatness of Churchill with the dishonesty of BDS.

6.9.2020 12:21am - (Replying to @PHuenermund and @JaminSpeer) The only thing I do not like about the US anthem is that it is almost always sung by a PERFORMER, not by the audience. Have you noticed? To me, growing up in Israel, this is more than strange, it defies the very meaning of "national" as well as of "anthem".

6.8.2020 6:15am - (Replying to @mddimick and @DaveBrady72) Sorry, I've never heard of "invariant contextual factors". Need a toy example (my weakness) no philosophy.

6.8.2020 6:11am - (Replying to @julianschuess and @DAGophile) I must confess that despite many efforts in the past I never truly understood what the "settable" system is, or what it offers. But Karim is still around, at UVA, econ. Perhaps he can explain it to us, mortals, and help us elevate econ. one notch up the ladder.

6.8.2020 4:56am - Here is another proof that a country condemned for existing can't be dogmatic. 39 years ago today, colleagues came to my office - indignant: How dare they do that? Unprovoked! etc. Yesterday I called my sister in Tel Aviv -- she and hers are alive! Optimistic. Worried about us!

6.8.2020 4:21am - @DeepMind interest in causal inference may signal a brighter future for data science. And @csylviav suggestion to start with #Bookofwhy, then proceed with Primer, has the potential of elevating many folks to Rung-2, then Rung-3. Happy sailing.

6.8.2020 2:26am - I think adopting the illusion that there IS an objective reality out there is extremely important, else we end up doing psychology or meta-psychology instead of science.

6.8.2020 2:14am - I have a strong hunch that Israel will take a lead in re-directing "data science" to its right course. After all, a country condemned for existing can't afford to be dogmatic.

6.8.2020 1:18am - Good point. We should change "data-assisted-science" to "data-intensive-science".

6.8.2020 1:00am - Agree. To be a "science", "data-science" should read "data-assisted-science". The way it reads now, "the science of data" has little to do with science, it has turned into "computer-assisted-statistics" or "statistics on stereoides". Yet donors love building those tall "Centers."

6.7.2020 9:50pm - (Replying to @eidlin and @DaveBrady72) My 2 cents. We must define things before we discuss how to estimate them. The hypothetical experiment of re-staffing police with color-blind cops is WELL DEFINED (removing an edge in the graph). Once we do that, we discuss what data/experiments are needed to assess its effect.

6.7.2020 9:37pm - (1/ ) Yes. You will be able to use "automated Machine Learning flows" to go from samples to distributions, then use causality to go from distributions to answer questions that cannot be answered by data alone, no matter how clever the learner. Awareness of the "no matter" is crucial
6.7.2020 9:37pm - (2/2) and it is THE mental block that keeps ML folks from seeing the necessity of causal modeling in tasks requiring more than data-fitting (Rung-1). I hope this road block disappears by the time you finish the book, and you'll then find yourself ahead of 90% of your ML peers.

6.7.2020 4:16pm - A word of caution. The statement below is about what data-science ought to be, not how it is practiced today. If you visit any of the newly erected "data science centers" you'l find that the taller the building the less likely you are to find a researcher who interrogates reality

6.7.2020 4:07pm - (Replying to @desai_pratik) My views are simple: We have so much philosophical soul searching to do, especially in the age of computers - the first model of human thoughts -- that debating what Hegel believed or did not believe seems a waste of time.

6.7.2020 3:47pm - (Replying to @luislamb) It just occurred to me. In computer science, we start with fundamental laws and go to the corollaries. In Epi, so it seems, folks go the other way; starting with phenomena such as collider bias, then proceed towards unifying principles. There must be wisdom in both approaches.

6.7.2020 3:32pm - (Replying to @resourcefulco) Kudos to the Bristol group for writing about collider bias even before COVID-19. Next conquest should be d-separation.

6.7.2020 2:37pm - One positive aspect of COVID-19 pandemic has been rising attention to the fundamentals of causal reasoning, in this case: collider bias. Yet rather than surrendering to bias, it should be coupled IMO with methods of RECOVERING from selection bias, eg

6.7.2020 2:12pm - (Replying to @AGI_programmer) I should have said "interrogating reality" or "interpreting data", as said in #Bookofwhy, or, more accurately, "using data to interrogate reality". Thanks for the pointer.

6.7.2020 2:01pm - For those who ask, I must add that my views on history of thought, hence on debating about ancient philosophers beliefs, is fairly whiggish, as summarized in this (unpublished) paper with Dana MacKenzie.

6.7.2020 12:21am - (Replying to @HenMazzig) You are right, sister Linda, "people of certain lineage" are saying explicitly that you ain't fit to lead any progressive movement, see, not because of your "lineage" but because you are a Zionophobic bigot, something you cannot bring yourself to deny.

6.6.2020 11:39pm - Titled: What's Sex Got to Do With Fair Machine Learning?, this paper will probably generate lively discussions. It's hard for me to see though how "constitutive diagrams" would tell us how the social meaning of a group would be changed.

6.6.2020 10:50pm - An interesting paper on how people judge causal responsibility of agents and events Note: I find the proposed "robustness-based" criterion hard to grasp, perhaps because it was not defined symbolically, as did causal necessity and sufficiency.

6.6.2020 10:11pm - (Replying to @EmraniMd and @PeterBeinart) Yet Linda Sarsour is not invited by Hillel Directors to speak on US campuses and sanctify their discomfort with Israel, as they did with @PeterBeinart on my campus in 2010. See "Jews of Discomfort"

6.6.2020 9:59pm - (Replying to @zevkalman) I was already 31, but the traumatic experience of anxiety, indecision and material fear of those few days have left such a lasting mark on our collective psyche, that it is still shaping our perception of what peace arrangement should prevent from happening.

6.6.2020 9:38pm - (Replying to @primedivider and @lexfridman) I second! Whoever is shocked by common sense deserves a transcript. I can use one too, though I am no longer shocked, just tolerating.

6.6.2020 9:10pm - I love philosophy, but what puts me to sleep is when philosophers start arguing about what another philosopher believed or did not believe.

6.6.2020 8:35pm - (Replying to @lmonasterio) I had a slight suspicion that you are referring to Richard von Mises whom I know from my philosophy reading. I can even see his book "Positivism" smiling at me from the bookshelf. But I did not realize he means so much to people in Brazil. Thanks for telling me.

6.6.2020 4:57pm - (Replying to @DaveBrady72 and @doc_thoughts) Such claims are not coming from counterfactualists but from Potential Outcome disciples (eg Holland 1986, Imbens and Rubin, 2015) who never snapped out of the outdated "treatment assignment" conception of causation. See

6.6.2020 4:10pm - As much as I dislike military parades and army uniforms, today, on D-Day, my head bows down at the sight (below) of these young men going into a battle they knew had to be won.

6.6.2020 7:06am - (Replying to @iGabrielX360) Half the world will never forgive Israel for choosing Life over Love. Which, as a student of human thought I perfectly understand; nothing could be more insulting than rejecting one's love. Luckily, the other half admires Israel for saying NO to postmortem love.

6.6.2020 6:11am - (1/3) Today marks the 53rd anniversary of the Six-Day War. A war that has left three deep scars in the collective psyche of all Israelis. First, the scar of ANXIETY on the days leading to the war: Is Nasser about to do what he has been promising for 19 years -- annihilation?"
6.6.2020 6:11am - (2/3) Second, the euphoric scar of SECURITY: We are alive and free to actually walk the paths we took so many times as children since we learned the Aleph-Beth. Thirdly, the scar of the Arab Summit (Khartoum, Aug 29, 1967) and its devastating "THREE-NO's" resolution - No peace,..
6.6.2020 6:11am - (3/3) No recognition, no negotiation with Israel -- which gave birth to the settlement ideology: "If we are destined to live by the sword to the end of days, we better do it from a position of strength, not at the mercy of guns or rockets." We are still carrying these three scars.

6.6.2020 5:11am - (Replying to @Claire_Voltaire) Well spoken, @Clare_Voltaire. The Chutzpah of some anti-Zionists may exceed their hypocrisy, see: And @PeterBeinart is master in ennobling this hypocrisy among "Jews of discomfort," as I observed here: Shabbat Shalom.

6.6.2020 4:06am - (Replying to @PeterBeinart) Excuse me, but on my campus it's the anti-Zionists who are waiving the "we aren't anti-Semitic" card to avoid debate of their moral deformity, as if Zionophobia is a Kosher type of bigotry, entitled to automatic free-speech pass, unlike Islamophobic or white supremacist speeches

6.6.2020 3:19am - (Replying to @DrIanKellar @SGSmith_87 and 2 others) I am glad to have been a catalyst for the clash between Matthew and his students/friends but the idea of not knowing "what he was talking about" strikes me strange. If he was talking causal inference in 2004 he must have known what he was talking about.

6.6.2020 2:51am - There are so many enlightened, learned and globally-minded colleagues to whom I would like to send this Tweet that I have decided to retweet it w/o a comment, just a humble mirror of their moral deformity.

6.5.2020 5:21am - What is badly amiss in our academic ivory towers is for colleagues of conscience to immediately speak out and publicly expose Ziono-phobic bigots for what they are.

6.5.2020 5:04am - Metascience: the science of doing science: The reason I think this collection will be of interest to readers is that the adventurous journey causality has taken in the past century is a microcosm for many of the issues discussed:

6.5.2020 4:15am - This is a nice introduction to graphical causal models for energy research folks, who study the effects of various intervention programs on household energy saving behavior. It provides a fairly comprehensive summary, and even touches on Simpson's paradox.

6.5.2020 12:05am - So true!. Ten years from now, historians will ask: How did scientific leaders of the time allow society to invest all its educational and financial resources in data-fitting technologies and so little on data-interpretation science?

6.4.2020 9:29pm - (Replying to @eliasbareinboim @PHuenermund and 13 others) Congratulations, Paul! And dont forget your comrades, still in the trenches.

6.4.2020 1:55pm - (Replying to @neuro_data @KordingLab and 2 others) Overslept - missed it - sorry. This post represents my current position on how healthcare leadership is missing on AI Feel free to re-post to participants.

6.4.2020 1:26pm - (Replying to @julianfdezme and @bradheath) A sample of 1 shows no correlation.

6.4.2020 1:24pm - (Replying to @tmorris_mrc and @stephensenn) Will have a look. Her examples are very clear.

6.4.2020 6:03am - (Replying to @HenningStrandin) I hate to consider myself "radical" though when it comes to philosophy, I dont mind. I ask different questions than Lewis and Hume: (1) How come people form consensus on counterfactuals much before causation? (2)How would a robot understand our natural language - counterfactuals?

6.4.2020 5:07am - (Replying to @stephensenn) In the simplest setting, we have n units, one treatment and its compliment (placebo), and we send k randomly chosen units to treatment and n-k to placebo. The calculus looks at the outcomes of all units and outputs (1) An estimate of ATE (2) Standard deviation. Am I getting it?

6.4.2020 4:41am - (Replying to @stephensenn) I think I am getting closer to understanding. Just some clarifications: (1) What is "block & treatment structure" (2) estimates of what? As Americans say "I am almost there".

6.4.2020 2:08am - (Replying to @stephensenn) I have tried several time to read your descriptions of Nelder's calculus but I am missing the basic vocabulary: eg. structure, block structure, design, balance, etc. Can you translate to non-experts like me? Perhaps in a form of a simple question that the calculus can answer?

6.4.2020 12:09am - (Replying to @desai_pratik) Try this long link: It takes me straight to the Microsoft lecture, Paris, 2013.

6.4.2020 12:04am - (Replying to @stephensenn) I am inclined to believe it but, so far, I have not been able to grasp what this calculus does, namely, what information it takes as input and what conclusions it delivers as output. I have been bought up to think in input-output terms, an incurable weakness which served me well.

6.3.2020 10:21pm - (Replying to @11kilobytes and @Kirsten3531) As strange as it may sound, I have not changed my mind.

6.3.2020 10:18pm - (Replying to @luislamb) Thanks. It is always invigorating to listen to a younger speaker, and a friendly one, too.

6.3.2020 3:24pm - Here is a better link to my Microsoft lecture It goes straight to the talk w/o commercials.

6.3.2020 4:52am - (Replying to @NandoDF @ProfBrianCox and @d_spiegel) This is precisely what I am worried about, that politically motivated actors will hijack the causal revolution, while our science leaders, both in stat and ML will be watching from a distance and continue the data-fitting gold-rush unquestioned.

6.3.2020 4:33am - In my Microsoft lecture I talked about statistics as the "calculus of shadows" and causality as a 3D calculus. I can think of several 4D's and 5D's, like object-based and agent-based calculi. But we first need to overturn th tyranny of statistical thinking

6.3.2020 4:14am - (Replying to @Bertrand_allen2 and @lmonasterio) I disagree with Hitchcock (author of, counterfactuals come first, causation second. The former is consensual, the latter debatable.

6.3.2020 3:36am - Homework solutions to the exercises here: will gladly be provided if you write to I just looked them over, the intellectual content compiled in this homework deserves a book of its own. Anyone volunteers as a co-author?

6.3.2020 3:21am - Glad you asked. I have been in this business for 3 decades and have not found anything clearer, more rigorous or friendlier than Chapter 4 of Primer. Here is a free link Enjoy, and don't let the mystics tell you counterfacuals are "hard".

6.2.2020 5:00am - (Replying to @19Naranjito82) I know nothing about antitrust, mergers, etc. So, I cant really be of much help, except to suggest: What do you know about those issues that a super statistician does not. That's the beginning, put it down on paper, first in words, then in DAG, the rest will follow.

6.1.2020 11:07pm - (Replying to @FarihaAnna) Debugging is where Bayesian networks hatched. Glad to see DAGs returning to debugging. But being unfamiliar with the systems being debugged, makes it difficult for me to generalize the proposed method.

6.1.2020 10:50pm - (Replying to @tytung2020) An intervention can be either external or internal, it means a "change" relative to the way things normally behave. An earthquake is external to my everyday life, but internal to geological dynamics. This paper may be helpful, an interpretation of do(x).

6.1.2020 8:53pm - (Replying to @wexlerwriting) Sanity demands an illusion of normalcy.

6.1.2020 4:15pm - An interesting paper on causal discovery using both observations and interventions The problem is to recover a causal graph while minimizing the cost of interventions.

6.1.2020 3:13pm - (Replying to @kavlak_batuhan and @dlmillimet) Speaking of agriculture, I've just received this paper on Mango tree growth: Hope it is of help.

6.1.2020 4:33am - My wife Ruth remembers that day, 6 yr old, Baghdad, 1941. Her mother forbade her to look out the window, she did anyhow and saw the bodies, in puddles of blood, as the stores across the street were looted, and the mob roared. The mob still roars. Not in Baghdad, in the USA!

6.1.2020 3:24am - Pleased to inform: The Supreme Court of Pakistan has deferred hearing our appeal of the Sindh Court decision that would free the four men convicted in the kidnapping and murder of our son, Daniel Pearl. We remain committed to pursuing our appeal and realizing justice for our son.

6.1.2020 12:36am - (Replying to @ZionessMovement) I was wondering what reception you had in the #BlackLivesMatter demonstration, and whether it was better than the one in the Women March. Did they allow you to carry Israeli flags?

5.31.2020 12:42pm - What do mainstream statisticians want to know about causal inference and The Book of Why? This interview with David Hand unveils part of the puzzle: And given that statistical thinking still dominates ML research, the answer may be of interest to many.

5.31.2020 11:03pm - (Replying to @PLetouze) Once you talk to computer scientists you can also compare assembly language to C++. And, to engineers: a wave form vs. its Fourier transform. I once wrote a note on it, let see,... here it is: I still enjoy reading it. Take a look.

5.31.2020 10:55pm - (Replying to @kavlak_batuhan and @dlmillimet) Nothing comes to mind, sorry. But if I had a problem in agriculture, I would try solving it the easy way: what do I know, what I want to know, what data can I gather. Done. The rest is sheer logic. I bet your solution will turn out better than what you can find in the literature.

5.31.2020 10:40pm - (Replying to @lqpeng @TimHarford and @FinancialTimes) Agree. @TimHarford should review @Bookofwhy for the innocent readers of @FinancialTimes

5.31.2020 10:32pm - (Replying to @mark_learns_sci) The trouble is that some grown ups who could find Israel on some maps are pretending to fall for these juvenile slogans and read them as saintly messages of peace.

5.31.2020 1:43pm - Some readers complain that economics was not mentioned in my tweet. Agree, econs have different kinds of questions, which we can see in @dlmillimet post: "Let's turn to the regression model so we can use some econometric". Plus, I wonder if they would interview an outsider.

5.31.2020 4:54am - (Replying to @yudapearl and @No_W_A_R) No country has ever been given sovereignty while calling for the obliteration of its neighbor. This is no paranoia but commonsense. Sovereignty entails the means to import arms, including missiles, and to accomplish what you have promised your children for 72 years.

5.31.2020 4:42am - (Replying to @No_W_A_R) I may be weak in Arabic but I know something about cause and effect. I know for fact that ALL my friends in Israel have been tuning their tallest antennas into Palestinian classrooms and media, yearning desperately for an inkling of acceptance - 72 years of nada. Thats THE CAUSE

5.31.2020 3:11am - (Replying to @No_W_A_R) Intentional laws, resolutions and declarations are always predicated on mutual acceptance and end of hostilities. As you know well, no Palestinian leader, cleric or educator has ever accepted Israel's existence (in Arabic!). To every Arabic-speaking person this means "no-Israel".

5.31.2020 1:20am - Can colleges launch data science programs fast enough? via @EdDiveK12 What does it take to tell them that data without causes ain't a science but statistics on steroids. I said so here: and am even more convinced now.

5.31.2020 12:48am - (1/3) Reread Keele etal's paper on "causal interpretations" and I agree with their view of how the question is mishandled in the Political Science literature. Readers seeking a quick answer, the solution is given in Theorem 5.3.1, p. 150 of Causality. I am
5.31.2020 12:48am - (2/3) surprised that Keele etal do not name the criterion, "single-door", which accurately and painlessly tells you when a parameter can be estimated by regression. It is also a question that economists often ask, can't answer, and refuse help. See why:
5.31.2020 12:48am - (3/3) @dlmillimet 's post also reveals why economists have such hard time answering their own questions. This says it all: "Let's turn to the regression model so we can use some econometric". They know that regression says nothing about causation, yet DAG-less habit intoxicates.

5.30.2020 11:35pm - (Replying to @No_W_A_R) I beg your pardon, but I am the last to connect anti-Israelism to anti-semitism. I am calling for treating the former as separate, more dangerous form of racism, whose aims and consequences you know very well, since they preceded the creation of Israel -- I was there.

5.30.2020 11:14pm - (Replying to @fields_unsown) Indeed. Anti-semitism is a disease of an unfortunate up-bringing. Anti-Israelism is a calculated genocide. The former is an emotional dislike, which everyone likes to abhor; the later, an eliminationist design which some take very very seriously, including some dear neighbors.

5.30.2020 10:46pm - Vandalism is never OK. Antisemitism is never OK. But anti-Israelism is the lowest.

5.30.2020 8:26pm - This post reinforces my conviction (Causality p. 135) that the question whether a parameter has a "causal interpretation" has no causal interpretation, through Keele etal's paper may help us decipher what authors are really asking by asking it.

5.30.2020 6:09am - Congratulations to Scott and his graduating class -- the FIRST ever high-school to master causal inference! These kids are heading to college and I don't envy their stat-101 professors; they would not let them get away with causality-blind cliches. They will demand substance.

5.29.2020 12:28am - I am not familiar with #AIOps, but I strongly resonate to what you say about #ML and #DL researchers. Climbing up the ladder of #Causality is not in their DNA. Even when they wish to do #CI, they just can't see the ladder. Not their fault; it ain't in their textbooks.

5.28.2020 3:17pm (Replying to @strangeloopdave and @mishtal) There were many stories of SS officers showing a nanosecond of humanity before returning to savagery. My aunt's point was that it was direct eye contacts which triggered those nanoseconds. American Jews often forget that self respect is what sustains the respect of others.

5.28.2020 2:35am - (Replying to @LebiedzMarcin) The new version is saved as PDF. See url: short url: If it works, please tell others.

5.28.2020 12:06am - (Replying to @mishtal) My aunt, who survived Auschwitz told me a lesson I will never forget. Inmates who fell to the SS officer's feet and begged for their lives were sent to their death, and those who looked him straight in the eyes were (often) spared.

5.27.2020 9:06pm - Current @AAUP 's leadership actually views Zionophobic racism as a cute form of academic "activism". It is members/donors outrage that will force them to resign and let adult educators advance the cause of higher education.

5.27.2020 4:16pm - For readers who had trouble opening the ppt slides of "The Silent History of Cause and Effect", here is a post-treatment version, surviving a interventional surgery. I hope it works.

5.27.2020 5:56am - (Replying to @jonasobleser) Try any of these: url: short url:

5.26.2020 11:46pm - Will try but, truthfully, I get impatient and frustrated listening to audience, moderators, even reporters not asking the questions I wish to ask. @benmartel97

5.26.2020 11:35pm - History-minded readers may also enjoy, which describes how statistics, lacking the necessary mathematical tools, managed to cope with increasing demands for policy evaluation from observational studies in the years 1955-1980. A tribute to William Cochran.

5.26.2020 3:57pm - (Replying to @wokebeartov) Sorry to admit ignorance but I've not heard about Friston's FEP. What is it (in Twitter language)? And I am not sure causality is a "new Bayesian idea" if you consider my observations in "Why I am only half Bayesian":

5.26.2020 1:35am - Economists tell us: If you have a good instrument, you dont need a model ( This new paper asks: What if you suspect selection bias? I don't know how a model-free economist would begin to derive the results obtained in this paper.

5.25.2020 10:22pm - (Replying to @yudapearl @scmbradley and 2 others) I was right! Gong and Meng totally miss the paradoxical element of Simpson's paradox. They attribute the surprise to incomplete specification of a probability function, forgetting that even with complete specification the surprise comes and goes depending on the background STORY.

5.25.2020 7:08pm - (Replying to @KambizzzKhan) "You are not expected to finish the job, but you ain't free to quit trying" (Mishna, Pirkei Avot 2:21)

5.25.2020 3:59pm - (Replying to @danijarh) The two options will forever be competing for our conception until we find a clash between the logic of causal modeling and empirical observations. For example, a violation of the inequality P(y_x) > P(x,y) [Causality, Eq.(9.33)]. So far, I have not seen a clash reported.

5.25.2020 2:19pm - (Replying to @MaartenvSmeden @Richie_Research and @matherion) It is not "easy", but it is the "shallowest" form of knowledge we can expect about mechanisms. As to COVID, see Why it is belated? Because we still do not have a "Causal Science Center" to prepare society for the next Tsunami of information confusion.

5.25.2020 2:10pm - (Replying to @scmbradley @fitelson and @olehjortland) I havn't seen it, thanks for the pointer. But considering that it was authored at Harvard-Stat, my guts tell me to expect the same consistent denial of the C-word as we have seen before (see, namely, round-about denial of the logic of human thought.

5.25.2020 4:29am - Per readers requests, I am happy to share the colored ppt slides of "The Silent History of Cause and Effect": Silent histories deserve colorful showcase. Enjoy.

5.25.2020 12:31am - Last week I had the privilege of celebrating a virtual Muslim-Jewish Interfaith Iftar, renewing my 2003-2010 dialogue with Islamic Studies Professor Akbar Ahmed. Here's a video of this communal celebration, with 25 inspiring interfaith activists:

5.24.2020 7:09pm - (Replying to @KristinaLerman @bgoncalves and @fitelson) I take it that if your data is found clean of Simpson's reversals you would be able to do 1) generalization, 2) explanation, 3) fair predictions with full confidence. My theory tells me that all 3 tasks are hopeless w/o causal models Where did I go wrong?

5.24.2020 3:46pm - (Replying to @bgoncalves @fitelson and @KristinaLerman) Before we spend the energy of identifying Simpson's reversal in very large datasets we need to sit down and ask "why"?, what would it tell us if we find such a benign pattern in the haystack. I don't see the answer in Lerman's paper.

5.24.2020 3:28pm - (Replying to @bgoncalves and @fitelson) I believe that if one digs deeply into the agendas of the philosophical and psychological approaches one would find them computational in disguise.

5.24.2020 3:15pm - (Replying to @fitelson) Purely probabilistic analyses, on the other hand, tend to reinforce the "probabilistic causality" language which some (admitted fewer and fewer) philosophers still cling to. I marvel the reasons for this clinging here I was once there, among the clingers.

5.24.2020 2:58pm - (Replying to @bgoncalves and @fitelson) If you really like debates, these two blog postings and will give you ample sports for the money. But as I always say: One toy example is worth 100 debates.

5.24.2020 2:47pm - (Replying to @lewbel) Glad you liked the slides. I thought you would scold me for asking whether Haavelmo's legacy will survive the two attempted hijackings.

5.24.2020 9:53am - (Replying to @roydanroy @guyvdb and 2 others) I would expect so too. But it did not happen in those centers that I've visited, and I now understand why.

5.24.2020 8:45am - I don't believe such corruption would last one day had members and donors of @AAUP known what is done in their name. I therefore retweet it to readers, in case some are still members in this hijacked organization, so called "American Association of University Professors".

5.24.2020 8:19am - (Replying to @AaronCarine @elderofziyon and 2 others) Those who tried to find consistent logic in Zionophobic rhetoric have given up long time ago. The one logic that I've found is: Smear, Malign and Lie. That's how they win the hearts of the ignorant and the gullible, for example, the dpt of Anthropology at my university, UCLA.

5.24.2020 3:46am - As a token appreciation to readers of #Bookofwhy, I am sharing the slides I used in a seminar titled: "The Silent History of Cause and Effect" given at UCLA History of Science Reading Group. Feel free to use any of these historical-philosophical snippets:

5.24.2020 3:15am - (Replying to @fitelson) Purely probabilistic analyses, on the other hand, tend to reinforce the "probabilistic causality" language which some (admitted fewer and fewer) philosophers still cling to. I marvel the reasons for this clinging here I was once there, among the clingers.

5.24.2020 2:57am - (Replying to @ArtisanalAnn @eliasbareinboim and 4 others) I wonder why? I never met Elon Musk, so I am wondering what dangers you are alerting us to.

5.24.2020 2:51am - (Replying to @PHuenermund @conjugateprior and @beyers_louw) What are those "coefficients for Z"? Is that how the typical econ/management/social science folks talk? W/o distinguising outgoing from incoming? w/o distinguishing structural from partial regression coefficients? Why exclude readers like me who is trying to study how they talk?

5.24.2020 2:45am - (Replying to @PHuenermund and @beyers_louw) First, why not clarify to readers what kind of coefficients you wish to exclude - I am still baffled. Second, are there more precise conditions than the single door criterion? Why not educate people when you got their attention; they will thank you forever.

5.24.2020 2:35am - These wise words of @RayDalio seem to be taken straight from the manifesto of the first "Causal Science Center". Hey, @RayDalio , would you be the Center 's first spokesman? Honorary President? Beneficiary? The algorithms you spoke about are ready to serve humanity.

5.23.2020 9:42pm - (Replying to @elprofe_ivan) I would start with a list of tasks that EVERY data analyst aspires to do and which require causal modeling. Seven such tasks are listed here: Formal, yet gentle introductions to these tasks are given in Primer: Good luck.

5.23.2020 9:31pm - Paradoxes that offend our intuition unveil the logic that governs our intuition, namely, the engine that is driving our minds. This blog describes how this logic varies with the causal story told. Highly recommended.

5.23.2020 9:12pm - An incredible production of "Jerusalem of Gold" ending the international celebration of Jerusalem Day. An on-line choir of beautiful angels on the background of the city that symbolizes peace, hope and the oneness of mankind:

5.23.2020 8:51pm - Abdulhadi's record of hate spreading activities is widely known. Therefore, her selection by @AAUP is a POLITICAL statement meant to encourage such activities in the future. The Tax Exempt status of @AAUP should be examined for compliance with its stated educational mission.

5.23.2020 8:33pm - Embroiled in choreographing Causal Science Centers I've almost forgotten that May 20 marks the 2-year anniversary of the publication of #Bookofwhy. Dana and I are grateful to all readers who contributed to our understanding that the book has made a dent on scientific methodology.

5.23.2020 6:51pm - (Replying to @eliasbareinboim) From my perspective, RL resides at Rung-1.5 of the causal hierarchy, laboring to cover Rung-2, but few RL researchers know about the Ladder, or where they are on it. Would you agree with this assessment?

5.23.2020 6:42pm - Readers wrote, and I agree: It is not the paradox that is stubborn but philosophers (and scientists). After getting your PhD and tenure viewing knowledge as a probabilistic engine it takes a bulldozer to make you accept reality: It is a CAUSAL engine that drives our minds.

5.23.2020 6:36pm - (Replying to @Benoit_Allen @tdietterich and @eliasbareinboim) Hat off. Though note that causal discovery is ONE component in the Causal Science scheme. What you do with what you discover is as important.

5.23.2020 6:30pm - (Replying to @tdietterich @neu_rips and @eliasbareinboim) This is what is happening right now when Data Centers justify their existence with a "causal" decoration. It will become a meaningful term when the scientific contributions of that Center are evaluated through a causal lens, defined by Causal Science Centers.

5.23.2020 5:33pm - (Replying to @elprofe_ivan) What about a data science (ie stat/ML) branch within a causal science center? After all, estimation is part of the Causal Engine, and Rung-1 inference is part of the Ladder of Causation. see #Bookofwhy and,

5.23.2020 5:08pm - (Replying to @richardtomsett @eliasbareinboim and 3 others) Nice beginning. Still, the time is ripe for a scholar who co-leads a special interest group in big data to deliver an introduction to ML and statistics at the University of Oxford "Causal Science Institute."

5.23.2020 4:57pm - (Replying to @tdietterich and @eliasbareinboim) That's not the point. All Centers brag: "We also do CI." Yet there is still not one Center for Causal Science where CI is declared THE focus of research and education and that generates a fraction of the number of data-centric PhD's that a typical DATA SCIENCE CENTER graduates.

5.23.2020 4:45pm - (Replying to @PACESconsulting) Yes, the fact Simpson's surprise disappears when you just change the causal story behind the data, not the data itself.

5.23.2020 11:50am - (Replying to @KateandPie) I perfectly see your point, and I see Ford's horse sh**g in every Data Science Center that I visit. That's the nature of a horse. Can the tiger change his skin? (Jeremiah 13)

5.23.2020 11:09am - I thought so too. So where are the Bill Gates, Jeff Bezos and Mark Zuckerbergs of today? Are visionary business leaders a thing of the past? I am not talking "causality research"; we have it. I am talking Causal Science Centers, to educate next generation Phd's for decision makin

5.23.2020 10:29am - (Replying to @ERMANigeria) Terrific question! I personally believe there should be a deep causal foundation to probability theory, but it's not well developed (IMHO). Funny thing, probabilists think such foundation is unnecessary, exception is Savage attempt to base statistics on decision-theoretic axioms.

5.23.2020 10:21am - (Replying to @anbarbazan) I have commented on Granger Causality here I have seen some sensible papers on CI in time series, but need to search for them.

5.23.2020 10:06am - In theory you are right but, in practice, these CENTERS either don't, or pay lip service to causal science. It's ALL in the name. Once it's named "CAUSAL SCIENCE CENTER" the director must live up to expectations, and "more of the same" turns inexcusable, even with a CI fig-leaf

5.23.2020 8:28am - Our well-intentioned funding agencies are partly responsible for this mis-balanced mis-investment. They all want & fund "explainable AI" "robust AI" "life-long learning" etc etc. forgetting that, once projects are run by traditionally-trained PI's they end up "more of the same."

5.23.2020 8:12am - (Replying to @yguangabroad) The proper set of variables for PS is one satisfying the back-door criterion. Some call it "backdoor admissible" or "sufficient set". It must be someplace in the slides.

5.23.2020 7:26am - It is for this reason that I feel uneasy seeing our society invest almost all its educational resources in "DATA SCIENCE CENTERS" run by good people mindcuffed by traditional statistical and ML paradigms. Not one "CAUSAL SCIENCE CENTER". A terrible long-term (mis)investment.

5.23.2020 6:42am - Readers wrote, and I agree: It is not the paradox that is stubborn but philosophers (and scientists). After getting your PhD and tenure viewing knowledge as a probabilistic engine it takes a bulldozer to make you accept reality: It is a CAUSAL engine that drives our minds.

5.23.2020 1:09am - Remember "Simpson'ts Paradox: The Riddle that Would not Die"? (See I've discovered another article, this time by a philosopher, attempting to do the impossible -- explain the paradox in probabilistic terms: A stubborn paradox!

5.22.2020 9:06pm - (Replying to @yudapearl @Jsevillamol and @causalinf) The "single door" criterion is described here, Theorem 2, together with many other tools applicable to linear systems. It is one of those basic tools that may lift economics education from its current mess.

5.22.2020 8:11pm - A lecturer who called me "white supremacist" was given an award for "leadership" that "transcends the division between scholarship and activism" by @AAUP , an organization claiming to "protect quality higher education." Read: @AAUPUC Dear Mr. Orwell, ....

5.22.2020 2:35pm - (Replying to @hangingnoodles) I like the metaphor: Statistics on steroids. May I borrow it?

5.22.2020 12:38pm - (Replying to @aquariusacquah) I like your figure 3, but I would reverse the arrows. CI is the umbrella that unifies stat., econ. and CS.

5.22.2020 12:26pm - Amazing! Yet with all this power and efficiency ML is still unable to tell us what to do in case we find a drug that's good for men, good for women and bad for a person, or how to handle any of these seven tasks:

5.22.2020 11:56am - (Replying to @Jsevillamol and @causalinf) You are solving a structure discovery problem. Identification assumes structure and seeks to estimate the parameters.

5.22.2020 7:19am - (Replying to @yudapearl @Jsevillamol and @causalinf) The simplicity of this problem entices me to add a couple of poetic versions that stone-age analysts couldn't handle: (1) What are the testable implications of your model? (2) What confounders can be added w/o spoiling identifiability? (3) What arrows can be added?

5.22.2020 3:15am - (Replying to @Jsevillamol) Fun problem, that @causalinf should include as homework: (1) Each parameter is identified using "single door". (2) Each mediated effect = sum of products along the mediated paths. In the stone age, economists solved such problems using matrix algebra. Today it's duck soup.

5.22.2020 2:31am - And for those who feel like singing "Jerusalem of Gold" with me (D minor please), lets go:
"And when I come today to sing to you and adorn your head with crowns, I am smaller than your youngest son, and the last behind your poets." Thanks.

5.22.2020 2:07am - (Replying to @Jsevillamol) Are all variables observable?

5.21.2020 9:23pm - My grandfather took me to Jerusalem when I was 3, I could not visit it again after 1948 (Arabs have funny definitions of "holy" "history" and "sharing"). I returned in 1967, my son was Bar Mitzva there, and every stone sings history to me. Happy Jerusalem Day - City of Magic.

5.21.2020 5:35am - (Replying to @Richie_Research @dingding_peng and @hardsci) The sex of an applicant to a given school can be a function of the education given to boys and girls in that community and of the kind of programs the school is offering.

5.20.2020 2:45pm - What makes Uruguay different than Venevzuela? Good judgement? Commonsense? Perhaps just good luck. I salute you, President @LuisLacallePou

5.20.2020 5:30am - (Replying to @mardelgiu @TimBartik and @causalinf) Well put; we all go for safety. But why WORRY about internal and external validity when mathematics has resolved these two issues? (eg

5.20.2020 4:53am - (Replying to @karlrohe and @omaclaren) Simpson's paradox is in the disputed territory between CI and Statistics. Discovered by the latter and resolved by the former. The part of CI that resolved Simpson's paradox is in AI; statisticians are still perplexed. See and the discussion around it.

5.20.2020 4:41am - (Replying to @dylAntoniazzi and @Richie_Research) On this educational chanel CI stands for Causal Inference. I did not realize the conflict with Confidence Intervals, I hope our statisticians will forgive the intrusion.

5.20.2020 2:11am - Anyone who believes that peace cannot be gotten by blindness to its obstacles should listen (below) to @EinatWilf describing her new book, "The War of Return", which bravely addresses the super giant elephant on the road to peace in the Middle East.

5.20.2020 1:36am - Roger Waters, the man who sang racism to music, finds himself banned from his band's website, lamenting his irrelevance, yet regretting none of his lyrics.

5.19.2020 11:57pm - This Workshop on “Explainable Logic-Based Knowledge Representation” deserves the attention of readers concerned with explainability of ML systems. Reading the abstract already puts you in a curious and humble mood.

5.19.2020 10:40pm - This paper is the first COVID-19 analysis I read that goes beyond data-fitting and tells society: Yes! AI can be trusted to extract meaning from data, and can do so rigorously and practically if properly directed and wisely supported.

5.19.2020 9:37pm - The enthusiastic reception of my slides and homework (800 likes) promises a bright future for CI in education. I went searching for the solutions to the homework. Eureka!! I've found them, and will make them available to any instructor who pledges not to spoil the fun for others.

5.19.2020 12:30am - (Replying to @causalinf) If my slides and homework would help, here they are:

5.19.2020 12:25am - Those who wish to teach a class on my Causality book would benefit from my slides and homework which I am happy to share: Merry teaching, it should be fun.

5.18.2020 12:33pm - (Replying to @TuanAIWS) Thanks for the advance warning, I hope I can tune in.

5.18.2020 12:25pm - (Replying to @mardelgiu and @TimBartik) Thanks, this kind of tasks happened to be my concern too. But look at me, I got sucked into theoretical work after realizing that they cannot be handled with conventional methods. How are non-theoretical economists coping with this realization? Are they using inadequate methods?

5.18.2020 12:11pm - (Replying to @jmorenocruz and @causalinf) Have you tried Primer? It was written to bridge between Causality and #Bookofwhy, and is supported by Daggity software and solution manual. And, beside, it's a truly good book.

5.18.2020 11:04pm - (Replying to @tdetonberry @MacroPru and @causalinf) Many of our readers will benefit from the extension, which will help their efforts to revitalize economics education.

5.18.2020 9:07pm - An Annotated Summary of #Bookofwhy. Milton Masur (MD) sent me this book summary: which I believe would be valuable for both new and veteran readers. It is more detailed than an typical book review and still captures the flow of the main ideas.

5.18.2020 3:23am - (Replying to @AvanteQuora @shriyamite and @TMoldwin) Agree, but this brings us back to the use of mathematical models. What makes an economist a better match-maker of models to contexts than a non-economist? And where does the former get the training and tools to be a good match-maker?

5.18.2020 3:17am - (Replying to @mardelgiu and @TimBartik) I was hoping to learn "what economists actually do" but the meat seems to be under pay-walls and/or high-talks, unless some economist can summarize it all in a typical example.

5.17.2020 3:07pm - (Replying to @TimBartik) Totally agree. And would probably agree more if given a description of what most economists do in fact do, what theoretical tools they use, where they get these tools from, and why amateurs (like me) would fail to accomplish what they do without those tools.

5.17.2020 3:58am - (Replying to @shriyamite and @nntaleb) This exactly why I object to calling statistics "data science", even when supported by "deep learning".

5.17.2020 3:17am - (Replying to @mardelgiu and @TimBartik) I was hoping to learn "what economists actually do" but the meat seems to be under pay-walls and/or high-talks, unless some economist can summarize it all in a typical example.

5.17.2020 12:14am - This brutal indictment of economics has nothing to do with our debate with dogmatic economists. It's in fact an indictment of all mathematical models. It's rich however with juicy quotes, historical anecdotes, and hard facts - worthy of our History Chanel

5.16.2020 11:37pm - Seconding our Pakistani attorney: It is a defining case for the Pakistani State.

5.16.2020 10:34pm - @RashidaTlaib would do well to read a page or two in @EinatWilf's new book "The War of Return", just to see how decent people manage to rise from tragedy, and others labor to perpetuate it.

5.16.2020 1:52pm - (Replying to @Jsevillamol) Thanks for sharing.

5.16.2020 3:36am - (Replying to @yudapearl @PWGTennant and 4 others) Sorry, there was a mix-up in the link. The correct link to my lecture on Epidemiology is this one: Please forgive.

5.16.2020 2:57am - (Replying to @PWGTennant @DavidEdmonds100 and 3 others) My outsider's perspective of progress in Epidemiology are described (with facts and dates) in this videoed lecture, which should also be of interest to readers of our History Chanel.

5.15.2020 11:01pm - (Replying to @fmg_twtr @kinggary and 5 others) Interesting! Why shouldn't conditioning on the PS be the same (asymptotically) as conditioning on those covartiates? Eq. (11.10) in states that they should be the same. Some colliders do NOT induce dependence among their parents, and PS is such a collider

5.15.2020 9:42pm - (Replying to @fmg_twtr @kinggary and 5 others) I am not sure this is the bias that @kinggary referred to, because he said: "even if you choose the right set of covariates". BTW, your graph model should define what is treatment and what is PS, etc.

5.15.2020 8:59pm - (Replying to @kinggary @LydiaManiatis and 4 others) I presume by "model dependency" you mean, dependency on the assumed distributional form of P(X| cov). This dependency is absent when we match directly on the strata of cov. Though other obstacles present themselves, eg. empty cells.

5.15.2020 8:45pm - (Replying to @theBjornErik @causalinf and @itaisher) Thanks for sharing this interesting approach to actual causation. Can you demonstrate the basic requirements for causation (realization, composition, information, integration, and exclusion) on a toy example like the Oxygen-match-fire story, as done in

5.15.2020 4:49pm - I think so too, despite being biased, and I do not see causal thinking dominate the conversation about #covid19 risk factors -- it should, especially when it comes to fusing data from multiple sources, as I argue here:

5.15.2020 1:25pm - (Replying to @FlorianWilhelm and @data_hpz) Sure; when bothered by unwanted ads customers imagine all kind of theories, eg. the seller is stuck with large inventory, or, the seller's profit margin are excessive, etc.

5.15.2020 1:15pm - This blog post aims to democratize debates about COVID-19 policies by demystifying the basic scientific questions that underlie the assignment priorities in health care.

5.15.2020 12:57pm - (Replying to @TuanAIWS) I wouldn't exactly call Ramsey an "thinker in AI", but he was definitely one of the clearest forerunner of subjective probabilities and the revival of Bayes statistics in the 20th century, which influenced the 1970-90 debate on how to represent uncertainty in AI systems.

5.15.2020 5:53am - More for our History Chanel, a new book on the life of Frank Ramsey, a mathematical genius (1903-1930) who, in 1926, defined subjective probability by the odds an individual would accept when betting on an outcome, thus anticipating VNM (1947) and Savage

5.15.2020 2:39am - @data_hpz . I believe the "Campaign Targeting" problem is the same as the "unit selection" problem, a counterfactual analysis of which is given in, combining both experimental and observational data. gives a dynamic visualization.

5.14.2020 7:01pm - Our Spanish speaking readers will be pleased to know that a Spanish translation of #Bookofwhy will be available next month and, moreover, the book was selected as "Libro del Semestre", see We will soon be "on the same page".

5.14.2020 6:18pm - Speaking of AI and Society, my attention was called (albeit belatedly) to this review of #Bookofwhy in AI&Society Journal: It is interesting for me to see how many of the book's issues are related indeed to society expectations of AI.

5.14.2020 11:39am - It turns out that the Guided Tour of AI can be accessed free, through this link, which should give our History Chanel readers direct access to the chapter on causation.

5.14.2020 10:49am - (Replying to @RaulMachadoG) Our works on IV have always assumed non-linearity, as in the instrumental inequality or non-compliance bounds Linearity was assumed in #Bookofwhy for ease of exposition only. As to citations, I cite all original contributors.

5.14.2020 12:54am - Friday, 4 pm, May 14, 1948, Israel declared independence. Wild dancing in the hearts, not in the streets; a day earlier Gush Ezion surrenders to the Arab Legion and 127 defenders executed; a day later we (12 yr olds) found ourselves bombed by Egyptian war planes. What a miracle!

5.13.2020 10:50pm - (Replying to @yudapearl @kinggary and 5 others) Infinite sample is not a matter of wishful thinking, but one of understanding where bias comes from. Asymptotic analysis, fortunately, allows us to separate two such sources: finite sample and wrong set of covariates. The former diminishes with bigger data, the latter does not.

5.13.2020 10:14pm - (Replying to @kinggary @LydiaManiatis and 4 others) The PS weaknesses that @kinggary unveils are orthogonal to those I explicate in The former are weakness of estimation, the latter of identification (infinite sample). Curious: what is "fully blocked randomized experiment" that you compare PS to?

5.13.2020 4:49am - Time for our History Channel. This new book is an encyclopedic bibliography of almost every paper published in AI (skipping pros and cons comments). The chapter on causality would be of special interest to our readers, esp. historians and bibliographers.

5.12.2020 1:39pm - (Replying to @JakeVigdor @causalinf and 3 others) Now I understand where the notion of "pay to publish" comes from. Unheard of in my corner. And let's not forget the "cite to publish" industry. That's how some journals become "high impact": The more you cite the guru, the higher your chances of a favorable review.

5.12.2020 5:43am - (Replying to @KetchupEconomi1 @sterndavidi and @causalinf) There is no "pay to publish". How come so many economists miss my tweets that the fee is OPTIONAL. I can't grasp the perception of "incentive to accept more papers". Are reviewers/editors paid in economics? Not so in JCI. Where do you get this perception from?

5.12.2020 1:19am - (Replying to @lmonasterio) This video made me wish to study Portugese. I hope you liked #Bookofwhy

5.11.2020 10:53pm - Another reason for submitting to JCI: Your idea-paper will be reviewed by idea-friendly scholars, and you won't spend 2-years fighting with causality-blind reviewers, compromising your causal vocabulary to pacify their dogmatic minds. That was the purpose for starting JCI (2013).

5.11.2020 8:56pm - (Replying to @petersuber) No press release, just rewording of the publication policy, after weeks of negotiation with the editors who insisted on making the fee optional, for all the reasons that @causalinf has mentioned. They should reword it, to make the optionality as clear as the word @DeGruyterOA

5.11.2020 6:12pm - Recommended Machine Learning e-lecture from Sydney Australia, describing causal inference as a framework for understanding ML, not the other way around.

5.11.2020 5:07pm - (Replying to @LydiaManiatis @CMichaelGibson and 4 others) I am happy to answer, if I only understood what the question is. You refer to "this technique" and I do not know what "this technique" is. Please describe the idea in a few words, so I can pass it through my mind's filter.

5.11.2020 1:31pm - (Replying to @LydiaManiatis @CMichaelGibson and 4 others) Can you describe "this technique" in conceptual terms? What does it do?

5.11.2020 6:38am - (Replying to @LydiaManiatis @CMichaelGibson and 4 others) The bias reducing power of PS matching is equal precisely to the bias reducing power of adjusting for the covariates that PS deploys. It is proven here

5.11.2020 2:01am - I am retweeting my last contribution in a long debate on the new JCI open access policy (no submission charge, and voluntary publication fee), in which the value of JCI publications to rigid tenure committees was questioned. I encourage all readers to submit idea papers to JCI.

5.11.2020 12:21am - (Replying to @causalinf and @dedubyadubya) Allow me to summarize my contribution to this discussion on the policy and quality of JCI by giving your followers a glimpse at the kind of research questions JCI is entertaining: Econ students will find it a friendly and inspiring amplifier of ideas.

5.10.2020 10:48pm - (Replying to @causalinf @hyperplanes and @dade_us) Correct.

5.10.2020 10:27pm - (Replying to @causalinf @hyperplanes and @dade_us) There are no submission charges, and article processing charges are voluntary. Those who can afford it, pay, and those who can't, just need to say so. And, again, considering the unique culture and ideas nourished by JCI, its a real bargain, unmatched by any contender.

5.10.2020 10:03pm - (Replying to @causalinf @hyperplanes and @dade_us) No, it's like Robin Hood social justice: Take from the guy loaded with fat grants and give to the poor un-tenured professor who has a small grant and big ideas.

5.10.2020 9:56pm - (Replying to @causalinf and @BitsyPerlman) JCI is a high quality peer-reviewed journal. While it cannot guarantee you tenure, it can assure you top reviewers who know CI and appreciate ideas. It can also guarantee you readers who seek CI ideas, knowing they cannot get them in journals run by tenure-minded authors/editors.

5.10.2020 7:16pm - (Replying to @StevePerfocus @eval_station and @BetterEval) As an engineer, I was spoiled into thinking input-output. What is the input to your analysis and is the output, assuming that it goes smoothly?

5.10.2020 6:57pm - (Replying to @causalinf) Scott, There is no submission charge, and the publishing fee is strictly voluntary. So what's all the commotion about predatory journals and tenure committees? Eager to understand.

5.10.2020 2:02pm - (Replying to @prisonrodeo) The submission fee is VOLUNTARY. We need to convince the publisher to highlight this option in capital letters.

5.10.2020 1:55pm - (Replying to @mukdal and @PeterGardenfors) I hope that paper of yours was followed up by many more. Has the belief-revision community been impacted at all by the causal revolution?

5.10.2020 4:39am - Readers will be pleased to note that submission fees for the Journal of Causal Inference will be on a voluntary basis only, meaning that authors will declare how much they are ready to pay, depending on their budget restriction. This, I believe, should achieve truly Free Access.

5.10.2020 3:14am - (Replying to @birkenkrahe) We have invited a new Associate Editor from Germany, who will start in January 2021. Good point.

5.9.2020 6:55pm - Readers of our educational channel would welcome this new paper: It argues that "the incapability of causal reasoning is the reason of DNN’s vulnerability to (adversarial) data manipulations." Remedy: Make adversarial factors explicit.

5.9.2020 6:37pm - A Review of Using Text to Remove Confounding from Causal Estimates: In other words, using Text as a noisy proxy for unmeasured confounders, as in A survey of on-going works and pending problems.

5.9.2020 6:18pm - Peter Gardenfors is a philosopher of mind with whom I interfaced in the hay days of "belief revision" (1996): This paper shows Peter embracing causal reasoning from his broad and interesting perspectives. Welcome @PeterGardenfors

5.9.2020 4:51pm - Users of SEM's are in the habit of estimating the overall fit between data and the hypothesized model. I've never understood why one would do that, when local fitting tests are available (d-separation). This paper seems to combine local and global tests.

5.9.2020 4:12pm - (Replying to @zevkalman) The Journal of Causal Inference is a peer-reviewed journal and, in this respect, is different from arXiv. Thanks for the comment on #Bookofwhy which leads me to ask: How are "network graphs" used in data analysis?

5.9.2020 4:36am - (Replying to @quantadan) I dont believe recognition of one person or another will rescue economics education from its stagnation. Perhaps the Nobel committee should suspend the Prize until such time when 50% of econ PhDs understand indentifiability and testability of (simple) nonparametric econ models.

5.9.2020 3:48am - Announcing new publication policy for the Journal of Causal Inference. The Journal has switched to Open Access. Articles are now published with unrestricted access for everyone, thus boosting visibility and impact. Authors still retain copyrights and there are no embargo periods.

5.9.2020 3:05am - (Replying to @halbfinger) David. "A new target: saving life" has no other interpretation but a defamatory contrast that you probably did not intend to make. You owe us an apology, see why:

5.9.2020 1:32am - (Replying to @birkenkrahe) Let us know if any snug.

5.8.2020 11:03PM - The NYT owes an apology to me and my generation, who were saved by IDF from a promised 'monumental massacre" (Azam Pasha, Nov. 1947), for insinuating, even jokingly, that IDF has any other mission but that of saving lives. Awaiting a public apology. Judea Pearl @yudapearl

5.8.2020 5:49am - (Replying to @sbuhai) Of course there is more to Econ than causal inference. But neglecting causal inference would be judged "20 years behind" by economists recalling the econ aims at policy making. And please, not "your" graphical model again; take ANY model that can do what graphical models offer.

5.8.2020 5:37am - Speaking about "explainable AI", this paper shows that, even in classification tasks, and even after agreeing on a Bayesian Network classifier, answering "why" is not a trivial matter.

5.8.2020 5:28am - (Replying to @sbuhai) Had the top journal in my field not encourage content that I consider important (ie, the content of Paul's slides) I would not consider it an insult to state so, but an invitation to alert the editors to their responsibilities. Can you name a paper using graphical models in ECMA?

5.8.2020 5:02am - (Replying to @sbuhai) Please read carefully. There is nothing in my tweet resembling *all*. It addresses "econ textbooks" and "econometrica" which are 20 years behind. And if forward-looking economists do not shake their establishment, they will remain behind. So why the anger when someone tries?

5.8.2020 4:05am - Recommended for economists and other readers who are asking: Where can we get an on-line course in modern causal inference? Not exactly a "course", but pretty close, and you won't learn it from econ textbooks or econometrika, now lagging 20 years behind the times.

5.6.2020 1:57am - I blush when people call me a "philosopher"; perhaps because to us, engineers, philosophy is just "poetry". Now that I have been called a "poet", I am more forgiving towards philosophers. Thank you, @amalkhan , that tree branch is still peeking at Danny's room, in expectation.

5.5.2020 2:24am - (Replying to @ilan_sinelnikov and @SSI_Movement) Congratulations!!

5.4.2020 10:22pm - #TheodorHerzl. And I almost missed his birthday! Here is his picture from a balcony in Basel, a picture that ignited hearts, moved masses, made deserts bloom, created a state, and saved my family from two infernos. What a balcony!!!

5.4.2020 4:43pm - (Replying to @zeemo_n and @AndrsMontealegr) The more I think about it the less certain I am about the boundaries between science, philosophy and artificial intelligence. Especially if the science you are doing is not conventional and the philosophy you are doing is driven by the puzzle: "So, how do people do it?"

5.4.2020 5:10am - Philip Dawid has written a comprehensive overview of a his approach to CI The do-operator is simulated by a decision variable in a Bayesian Network. The paper illuminates what can be done without counterfactuals, a topic of my paper

5.3.2020 8:41pm - (Replying to @EinatWilf) A tested way to uncover Palestinians' agenda to undo Israel is to give their intellectuals a stage and let them rant freely about Zionism and its quest for coexistence. The truth will come out immediately, on the first page, in perfect English, and in no ambiguous terms.

5.3.2020 5:27am - World leaders: Release imprisoned journalists worldwide - Sign the Petition! via @Change

5.3.2020 5:22am - (Replying to @eyad_nawar and @eliasbareinboim) Determinism disappears if the action is disjunctive, as in "paint the wall either green or red",see From the painter viewpoint the action is deterministic (one choice) but from the policy maker, it is stochastic, not knowing what color will be chosen.

5.3.2020 3:12am - There is a bit more to it than a language/calculus, it is a tractable algorithm that tells us, not merely if a query is estimable, but also how to estimate it and, if it's not estimable, what kind of additional measurements or experiments will turn it estimable. It was undersold.

5.3.2020 1:01am - (Replying to @ExogenyKarl and @eliasbareinboim) The paper you sited is one of my favorites, but I think you missed the punch-line: "economics students should now be able to solve the eight toy problems I posed in Pearl, 2013 (see Appendix A, Section A.2)." Do you think it has happened?

5.2.2020 10:30pm - Readers who have been shunning the do-operator for being "too deterministic" can now embrace its stochastic cousin, "soft intervention", formalized in a recent paper by Correa and @eliasbareinboim :

5.2.2020 5:32pm - (Replying to @agpatriota @guilhermejd1 and 4 others) It sure can be captured. E(Y|do(X=1)) = E(Y|do(X=0)) and E(Y|do(X=1),Z=0) =/= E(Y|do(X=0), Z=0) See Causality page 35-36

5.2.2020 4:57pm - (Replying to @RanaBil10485167 @AsraNomani and @UmarCheema1) Justice to Daniel Pearl will usher justice to all innocent victims of brutality and extremism.

5.2.2020 4:50pm - (Replying to @Taks87) We hope that justice to Daniel will usher justice to all innocent victims of extremism

5.2.2020 4:41pm - The parents of murdered Wall Street Journal reporter Daniel Pearl asked Pakistan’s Supreme Court to prevent the men convicted of his abduction and murder from going free via @WSJ

5.2.2020 5:30am - We wish to thank The Pearl Project @AsraNomani for leading the effort to realize justice for Danny, the Committee to Protect Journalists @pressfreedom for their spirited support of this effort, & the Wall Street Journal, for contributing half the legal fees needed for our appeal.

5.2.2020 4:52am - (Replying to @rameshfilms) Remember you very well, and the documentary too, with Christiane Amanpour @camanpour . Glad you are going to screen it again soon; the world needs a reminder.

5.2.2020 3:54am - Today, my wife and I filed an appeal to the Supreme Court of Pakistan to overturn the "acquittal" of our son's murderers. We ask all people of conscience to support our efforts to protect journalists from imitators of Omar Sheikh and his ilk. For details

5.2.2020 1:33am - (Replying to @ExogenyKarl and @RayDalio) Thanks, I'll try to refresh:

5.2.2020 12:44am - (Replying to @mribeirodantas @tdietterich and @StanfordHAI) I do not think the good people at Microsoft and Google have the institutional clout to overturn the deeply entrenched thinking of the data-fitting culture. Recall, it is quite traumatic to grasp the idea that "interpretation" means going beyond the data and that it is necessary.

5.2.2020 12:22am - Could @RayDalio be the visionary executive who will save ML from itself by establishing a "Center of Data-Interpretation" at the University of XYZ, to counter the "data fitting" hype? see If Microsoft and Amazon won't, why not Wall Street? Dead serious!!

5.2.2020 12:03am - (Replying to @matt_vowels) Strangely, I had a similar experience, and so did many readers. Doubly strangely, the authors of the right book write: "we have not found this approach to aid drawing of causal inferences." You can't aid those who refuse your aid, but you can aim to understand of the refusal.

5.1.2020 7:53pm - (Replying to @kerstingAIML @mitbrainandcog and 4 others) Which "we" are we disagreeing on?

5.1.2020 7:33pm - (Replying to @bjh_ip) Aggree, and I am working on it.

4.30.2020 5:55am - A great article by David Suissa which, although does not target Dianne Lob explicitly, provides an in-depth analysis of her mentality, and the mentality of other "Jews of discomfort", of whom I wrote here:

4.29.2020 11:12pm - (Replying to @KyleCranmer @tdietterich and @StanfordHAI) Simulators are causal, and every causal model (completely specified) can be used as a simulator. But I can't see how DL can act as surrogates for these models, since the inputs to DL are data, not the ropes behind the data, and the inverse mapping data-->ropes is not unique.

4.29.2020 1:00am - A fairly harsh indictment of AI: Why hasn't AI had more impact? I have asked my colleagues at @StanfordHAI the same question:, but I am not sure they took notice. People blame the noisy data; shouldn't AI outsmart the noise-makers?

4.29.2020 12:15am - (1/ ) (Replying to @mribeirodantas @tdietterich and @StanfordHAI) It will happen when an enlightened executive of Microsoft or Amazon, etc., will realize that data-fitting in addictive, and building another "data-science center" will only worsen the addiction, and by the time industry will need to switch to "data-interpretive science"
4.29.2020 12:29am - (2/ ) (Replying to @yudapearl @mribeirodantas and 2 others) there will be no one to hire for the job. Nadda. The enlightened executive will then say: Let's establish a "Data-Interpretation Center" at the University of xyz, and endow it with 0.01% of the resources now pouring into data-fitting centers. Get ready, it will happen soon.

4.29.2020 12:10am - And the eyes of the world turn again to the Supreme Court of Pakistan and ask: What message will it send to their sons and daughters?

4.28.2020 6:39pm - To readers who keep on asking: what's the secret of looking so good at 72, and of feeling so happy despite 72 years of siege, the answer is simple: Its plain 1978 years of waiting, that's all it takes, just 1978 years.

4.28.2020 2:23pm - (Replying to @YFeyman @raj_mehta and 4 others) ignorability is "as if randomized, given Z" and exclusion is "going only through X". Two independent restrictions that even seasoned IV experts can't conceptualize w/o graphs, see why:

4.28.2020 2:07pm - (Replying to @YFeyman @raj_mehta and 3 others) Honestly, this is the first time I hear the adjectives "clear" and "tractable" attached to the PO framework. It makes me extremely curious: can you perhaps show us how you judge the plausibility of a simple ignorability assumption, like @stuartbuck1 :

4.28.2020 1:47pm - (Replying to @raj_mehta @YFeyman and 3 others) Good point. I could imagine conversations like those we are witnessing on Twitter going on w/o the language of DAGs to facilitate them. Even devout antaginsts of DAGs are using expressions such as "there are many paths from X to Y" "going only through W" etc. It speaks a lot.

4.28.2020 3:46am - Another celebration starts today at sundown - Israel's Independence Day. Join me in celebrating the most inspirational miracle of the 20th century: A tribe of beggars and peddlers lifting themselves from the margin of history to become a world center of science, art and business.

4.28.2020 4:24am - (Replying to @richarddorset) Many oppressed peoples, Yazidis, Asyrians, Kurds, even Tibetians are inspired by the Israeli experiment and studying the secret of its creation. The secret: Jews were not a "nomadic tribe" after all, but a dormant nation, driven daily by the dream of returning to their homeland.

4.28.2020 4:46am - (Replying to @yudapearl and @richarddorset) For a good book on the creation of Israel, I'd recommend Benny Morris "1948", though it may not meet your criterion of "new state based on ancient claims". For my grandfather it was "old state based on daily dreams". He prayed 3 times a day "return us in sovereignty to our land."

4.28.2020 3:16am - (Replying to @maximananyev @analisereal and 6 others) I think we can generalize your observation to read: applied econ use graphical metaphors quite often, albeit shyly and tacitly (eg see textbook descriptions of IV), and the language of diagrams can improve their practice by encouraging them to make those intuitions explicit.

4.28.2020 1:05am - Congratulations to @PHuenermund who will be joining the Editorial Board of the Journal of Causal Inference (JCI). Paul's addition is an open invitation to all economists to embrace JCI as an effective publication for disseminating ideas across disciplines.

4.27.2020 9:25pm - (Replying to @amt_shrma @autoregress and 8 others) The assignment need not be "random". It can be by student's birthdate, or by results of an eyesight test, as long as it is not correlated with U and e_y.

4.27.2020 2:08am - Update on Israel Memorial Day. The sirens will sound today (Monday) at 8:00 pm Israel time (1 pm EST,10 am PST). Join me in freezing (1 min) for Dani Balkore, who went to fight five invading armies in May of 1948, and made it possible for my generation to live a semi-normal life.

4.26.2020 9:22pm - (Replying to @patrickkloesel and @PHuenermund) Based on my many conversations, the percentage is around 95%. And, more importantly, it is not that they do not understand DAGs, they are committed to not understanding them, expressed by: "we have not found them useful in our type of applications".

4.26.2020 4:40pm - (1/ ) (Replying to @stuartbuck1 @PHuenermund and 2 others) I confess to not knowing everything about "how economists think and reason about ignorability" because my knowledge comes primarily from published articles, which may not reveal the entire process through which authors "think". I would therefore be grateful to you if you could
4.26.2020 4:52pm - (2/ ) (Replying to @yudapearl @stuartbuck1 and 3 others) help me, and many other readers, fill in the key principles of that process, by sharing with us how YOU think about ignorability. I capitalized YOU because, economists may vary in their intepretations of the term. So, if you are willing, let us start with the simplest
4.26.2020 5:00pm - (3/ ) (Replying to @yudapearl @stuartbuck1 and 3 others) 3/ "conditional ignorability" statement, Y(X) || X | Z, where X is a treatment, Y the outcome, and Z a set of covariates that are candidates for making the statement true. Can you walk us through the process by which you would judge whether the statement holds in a given problem
4.26.2020 5:07pm - (4/4) (Replying to @yudapearl @stuartbuck1 and 3 others) or scenario, or a model, or a story, or whatever helps you discern whether the statement holds true or not. No need to be pedantic on issues such as "strong ignorability" vs. "weak ignorability", or how Z was chosen. Just examining the problem and deciding true of false.

4.26.2020 6:52am - Good catch. This link should work I hope:

4.26.2020 6:27am - I am in receipt of a comprehensive paper on the role of causality in vision Although I find the section on "causal Bayes Nets" a bit incoherent, there is a lot I can learn from it, especially the interplay with naive physics and object-oriented reasoning.

4.26.2020 3:41am - Caught me by surprise! But we must add that, with the exception of a few dragonic islands, the dragons were eventually overcome by angels of commonsense.

4.26.2020 5:52am - (Replying to @PHuenermund) Not at all. It was in perfect timing; just a few days after the angry dragons have issued a non-ignorable ultimatum: dragonize or perish.

4.26.2020 3:47am - 100th anniversary !!! And I almost missed it !! To readers who have not heard of the San Remo Resolution, or the Balfour Declaration that triggered it, I wrote an oped on the latter which would be illuminating to read on this historic day.

4.26.2020 2:28am - ECOL VS. ECON While economists are moving from models to field experiments, ecologists seem to be going the other way. This paper:'s_guide_to_developing_explanatory_statistical_models_using_causal_analysis_principles/links/5e8e72c9a6fdcca78901f151/Scientists-guide-to-developing-explanatory-statistical-models-using-causal-analysis-principles.pdf seeks the advice of causal models to "guide scientists in their quest to develop explanatory hypotheses for evaluation."

4.25.2020 4:09pm - At sundown today, siren sounds freeze Israel into a 2-minute silence in remembrance of its fallen - In May 1948, our next-door neighbor, Dani Balkore (19), kissed his family, smiled to us, and returned in a coffin two weeks later. I freeze for you Dani.

4.25.2020 2:46pm - (Replying to @LennyVds) I would not phrase it exactly as you did. I would say: The power of SCM is that you can compute the likelihood of EVERY counterfactual sentence. For example, Y would be less likely had X not happened, given that X actually did happen.

4.25.2020 2:27pm - (Replying to @aulderic @Liv_Boeree and @robertwiblin) Agree. The birth-weight paradox should not be classified as an example of "Simpson's paradox". The surprise in the two paradoxes is of different nature. Someone should fix Wiki, and add "not" to: "it can NOT be explained statistically".

4.25.2020 6:13am - Israel Yom Hazikaron National Online Memorial Ceremony 8:30EDT via @israeliamerican

4.25.2020 1:15am - (1/3) Here is a golden opportunity to learn something of lasting value during our COVID-19 hibernation. There have been dozens of articles, dialogues, discussions, even tantrum debates, laboring to compare DAGs and PO frameworks, mostly written by distant commentators, who could
4.25.2020 1:15am - (2/3) not expose first-hand what it takes to solve a problem from beginning to end using one framework or another. This link will do it for you, in just a few paragraphs, using a simple chain X-->Y-->Z. Once you read these paragraphs you will never
4.25.2020 1:15am - (3/3) be intimidated by an invitation to compare the two frameworks. You might even become curious as to why some people would prefer one framework over another.

4.24.2020 10:21pm - (Replying to @maximananyev @Jabaluck and 7 others) I am not a bit surprised. "front-door" is one of many models involving 3 variables and one latent confounder, so i am sure it came up naturally in many applications. The fact that it permits nonparametric identification, came as a shock to statisticians (narrated in #Bookofwhy)

4.24.2020 2:29pm - (Replying to @HannesMalmberg1 @Jabaluck and @maximananyev) You are beginning to see why I called it "undersold". But the gains are not merely educational, they are methodological and computational. Tasks that are intractable in PO (consistency, redundancy, testability, etc) suddenly become as easy as writing down an SEM. Sadly undersold.

4.24.2020 2:17pm - (Replying to @HannesMalmberg1 @Jabaluck and @maximananyev) When I see the word "design", I see a potential for an exciting PhD project: Let's take the mental model that gives the "designer" a feeling of "design", formalize it, and show the designer how to design things better. This was done in several key areas, and science progresses.

4.24.2020 2:07pm - (Replying to @autoregress @PACESconsulting and 4 others) Thanks for adding a new acronym to my arsenal. The missing-data literature refers to your CMAR as MAR. But there, "missing" means unobserved variabls in some individuals, not missing individuals.

4.24.2020 1:56pm - (Replying to @autoregress @PACESconsulting and 4 others) What is CMAR? Is it Missing Completely at Random MCAR, as used here If so, my answer is probably YES. Most non-parametric problems in economics now have solutions. But it was undersold. Eco. was not fortunate to have a leader saying: Look guys, Its Great!

4.24.2020 1:29pm - (1/ ) (Replying to @The_RickMc @PHuenermund and 3 others) This is an important point. What make us judge a treatment as "as-if random" when we have no coins nor lotteries? It is our subjective inability to conceive of a confounding mechanism. Take the price of beans in China and the traffic in LA. None is random.
4.24.2020 1:40pm - (2/ ) (Replying to @yudapearl @The_RickMc and 4 others) Each varies systematically in its own dynamics. Yet, unless you are a smart alec and argue that some future-broker in LA may panic and call his clients to rush to their cars, the two variables are judged to be unconfounded. Randomization is sufficient, but not necessary.

4.24.2020 1:20pm - (Replying to @PACESconsulting @gelbach and 4 others) My first sentence says that for decades Heckman's selection-bias correction was tied to strong parametric assumptions. Nothing wrong with it. But if we want to snap out of such assumptions we need the logic of non-parametric models. That logic was a "novelty", highly undersold.

4.24.2020 11:00am - (Replying to @PACESconsulting @gelbach and 4 others) New results after decades of stagnation do demand new tools. Specifically, the idea that selection bias can be reduced or eliminated without making parametric assumptions demands tools to handle non parametric models. Why the resistance to novelty?

4.24.2020 10:49am - Measurement errors are represented as proxy variables, as shown here . The do-calculus just operates on the graph after these proxy nodes are added. In most cases identification requires auxiliary machinery, as shown here

4.24.2020 2:24am - For the many who inquired on the latest in the movement against the "acquittal" of our son's murderer, this article offers a fairly comprehensive report. Thanking all readers for their help and empathy.

4.24.2020 2:21am - To our readers in Israel, The Israeli Association for Artificial Intelligence has announced a Doctoral Dissertation Award. Details are here:

4.24.2020 1:37am - (Replying to @teemu_roos and @UusitaloLaura) You are so right, skipped my mind and my search engine. It is indeed in Which, BTW, re-reading it 32 years later, is a damn good paper!. I don't recall if it was folklore. The firing squad was folklore (mentioned in I. J. Good)

4.24.2020 1:25am - From the day I was told that RCT's require "careful design", I suspected that RCT's experts must be using a mental causal model. Here is a paper that supports that suspicion: and shows experts what they can gain by making that mental model explicit.

4.24.2020 12:32am - (Replying to @Jabaluck @PHuenermund and 2 others) Luckily, I have not met a serious economist who thinks I am a charlatan. They know what you know, that my "guilt" is only in calling to make assumptions explicit in a language that is most meaningful to researchers (SCM) but became unfashionable in 1980, yet producing new results

4.23.2020 10:51pm - (Replying to @yudapearl @PHuenermund and 3 others) And there is a good reason why justification is avoided. Justification boils down to saying: "I cannot think of any factor that would affect both X and Y." It is tabooed on 2 counts: (1) It's dreadfully subjective, (2) It invokes"factors affecting," almost like graphical criteria

4.23.2020 10:43pm - (Replying to @PHuenermund @Jabaluck and 2 others) Moreover, students (and their advisors) reading the final papers get the impression that as soon as you say: "we assumed ignorability" and cite revered authors who also did so, you are a scientist. The ratio of such papers to justifying paper is 100:1 (based on mental counting).

4.23.2020 10:28pm - It is always a thought provoking exercise to imagine how the general public would react to issues that we have been debating on Twitter with such intensity. One thing is clear, causality is gaining stature as an independent species of scientific inquiry.

4.23.2020 9:49pm - (Replying to @maximananyev and @Jabaluck) You are almost right. Applied econ. will challenge every assumption that is explicit, but would rarely challenge those that hide under fashionable technical titles such as "ignorable", which hardly anyone knows what it means. It's the old peril of being honest and transparent.

4.23.2020 9:26pm - (Replying to @Jabaluck @autoregress and 3 others) What's the point? Creative people can always find a link between the price of beans in China and traffic in LA. If they believe the link is strong, put an arrow and continue. If its weak enough to ignore, remove the arrow and continue. W&C did the latter and failed to continue.

4.23.2020 9:10pm - (Replying to @Jabaluck @autoregress and 3 others) The only economist I met who thought "carefully about DAGs" was the late Hal White. If you know more, they were not here: when I asked them to answer a few elementary questions, like which parameter can be identified by OLS, or which model is testable.

4.23.2020 8:53pm - (Replying to @Jabaluck @autoregress and 3 others) There are many ways to justify absence of an arrow. For me, it is enough to say XWY are medical factors and Z's behavior is a bureaucratic one. Done. If we must cheer Angrist for his insight, fine, but we still need to decide if the query is identified.This is where WC fumbled.

4.23.2020 7:47pm - Great initiative, and a very informative window into the enterprise of "Evidence Based", which I never understood. Speaking of evidence from population to individual, the idea of combining experimental &observational data should illuminate the discussion:

4.23.2020 4:07pm - (Replying to @autoregress @gelbach and 3 others) If treatment Z is administered by a clerk who follows some protocol on how to respond to symptom W, then this clerk is shielded from the confounder U, which affects patients, not clerks. I'm sure you have clerks in economics?

4.23.2020 3:57pm - (Replying to @Jabaluck @autoregress and 5 others) But to assure consistency among a set of dependence-independence assertions, we must assume that they emanate from some, possibly unknown distribution.

4.23.2020 2:35pm - (Replying to @Jabaluck @autoregress and 5 others) This is the beauty of DAGs. You do not commit to a specific structural model. You commit only to the class that is compatible with your knowledge, namely, the source of variation of each variable. This is many time more meaningful than "which PO is ignorable conditional on whom"

4.23.2020 2:21pm - I surely read it, and commented on it here:

4.23.2020 2:12pm - (Replying to @Jabaluck @autoregress and 5 others) I have a friend who "understands" the connection between independence and probabilities but he would never put the definition X||Y iff P(X,Y)=P(X)P(Y) on paper, or in words. His papers are full of X||Y and X||Y|Z sentences but no mention of P. His disciples claim he "understands"

4.23.2020 1:25pm - (Replying to @autoregress @guilhermejd1 and 5 others) If "Yup" means "Yes" then we disagree on the 2nd "Yup". Imbens-Rubin-Angrist not only do not mention this connection in their books, they resist the connection like a plague. But I welcome your first "Yup", and hope that we can agree on its logical implications

4.23.2020 1:09pm - (Replying to @autoregress @guilhermejd1 and 5 others) I did not say "ARE" causal models. I said "emanate from" a structural model, or, "can all be derived from" a structural model. Do you buy that? Do Imbens-Rubin-Angrist-Peschke buy that? This is the 1st Fundamental Law of causal inference.

4.23.2020 12:48pm - (Replying to @guilhermejd1 @autoregress and 5 others) "Embrace"? You must be kidding. I guess the idea that potential outcomes are intrinsic properties of SCM has not been sunk in yet. Every SCM model assigns a probability to every conceivable potential outcome. Potential outcomes are rooted in SCM, and trees w/o roots wither.

4.23.2020 4:53am - (1/ ) I am retweeting this post, knowing that hundreds among our readers are intimidated by colleagues on the issue of "real-world" example. As if going to Somalia and conducting field experiments elevates you to a new level of understanding. The post demonstrates a "real-world"
4.23.2020 4:53am - (2/ ) problem that was mishandled by prominent investigators precisely because they dismissed the tools of causal inference. In my follow-up Tweet: I ask: How many "real world" problems are mishandled by those who talk "real world" to dismiss CI tools?

4.23.2020 1:48am - (Replying to @yudapearl @autoregress and 4 others) Now, Wermuth and Cox are not DAG fanatics. They posed this problem b/c it came up in their investigation of sequential treatments. They wrote 3 articles on this problem, which involves only 5 variables. What does it tell us about economists and their "real world" arguments?

4.23.2020 1:29am - (Replying to @yudapearl @autoregress and 4 others) More on "Real-world" herrings. I was asked repeatedly to give one "Real-world" example which do-calculus can solve, and which "real-world" experts could not. Here is one,, called "indirect confounding" by Wermuth and Cox, also discussed in #Bookofwhy p.241

4.23.2020 12:59am - (Replying to @autoregress @gelbach and 3 others) Every setting that you would be willing to call "real-world" can be solved by these problems, or through the general theorems that these problems illustrate, or labeled "unsolvable" by the completeness theorems that follow (in later papers)."Real-world" boasting is a red herring.

4.23.2020 12:45am - (Replying to @SylvainCF @pa_chevalier and 17 others) I am embarrassed to be on the same list as the great Hume. But, given what I say about him and about counterfactuals on page 266-9 of #Bookofwhy, perhaps he would accommodate my presence on the list.

4.23.2020 12:27am - (Replying to @Jabaluck) The trouble comes when you assume what you need BECAUSE you need it, not because you know it. And this indeed is the common case when PO folks assume conditional ignorability, save for RCT's, and some IV's settings, when natural lotteries are available.

4.22.2020 11:43pm - (Replying to @autoregress @analisereal and 3 others) How can you say that Pearl ignore monotonicity? See my discussion on monotonicity with Imbens Not only can SCM articulate monotonicity when it is plausible, but we can also test it when it is in doubt (See Causality). We do not shun tools, we use them.

4.22.2020 11:02pm - I like your crisp distinction between stating what one "needs" vs. stating what one "knows". It really summarizes the entire debate between the CI and PO frameworks. In CI we use structure to encode what we know, PO to encode what we wish to know, and logic to connect the two.

4.22.2020 10:47pm - (Replying to @analisereal @autoregress and 3 others) I have seen some. Imbens and Rubin (2015) call these identifying assumptions "The Science."

4.22.2020 10:35pm - (Replying to @autoregress @gelbach and 2 others) Puzzled. How could economists possibly have fought this or a similar war, if they did not have a 3-rung causal hierarchy in mind, nor complete tools for managing each rung. Can you point me to any similar war? Was "moving on" a resolution or "give up, lets do what we do best???"

4.22.2020 9:08pm - (Replying to @gelbach @guilhermejd1 and 3 others) Can you use Heckman's correction to solve any of the siimple problems solved here: ???. As I stated before. CONCERN - yes. Results - ??? And I dont blame you, new results demand new tools, which your students will have to acquire by independent reading.

4.22.2020 8:56pm - (Replying to @jeffreywatumull) No, this kind of assumptions have no testable implications and are needed as input to causal calculus.

4.22.2020 8:43pm - (Replying to @maximananyev @autoregress and 5 others) Can you summarize how @ecnomeager represents disparities between two populations.

4.22.2020 8:38pm - (Replying to @guilhermejd1 @autoregress and 3 others) Agree with you completely. Economists have indeed been "concerned" with these questions, as were most scientists since Campbell etal. But when you ask them to solve a tiny problem, eg you see how far behind they have allowed their "concerns" to take them.

4.22.2020 8:14pm - (Replying to @Jabaluck @gelbach and 2 others) Can we put aside me and my failures to engage. Let's return to economics "embracing" machine learning, and examine what precisely has economics "embraced" in the past 3 decades.

4.22.2020 8:09pm - (Replying to @autoregress @Jabaluck and 2 others) See my reply to Imbens who made similar points.

4.22.2020 8:06pm - (Replying to @Jabaluck @gelbach and 2 others) It is more than a question "easier". It is one of "intractability" which in practical terms means "impossible". And let's leave DAGs aside. I am insisting only on transparency, and would welcome a transparent alternative to graphical models.

4.22.2020 8:00pm - (Replying to @Jabaluck @gelbach and 2 others) I seriously doubt that you will that that flexibility. Your students will, but not those who grew up in the insular green house of 1990's econometrics.

4.22.2020 7:51pm - (Replying to @Jabaluck @gelbach and 2 others) Forget discovery. I want economists to do what they have been advertising to be doing since Haavelmo, before being hijacked by statisticians who believed that conditional independencies among counterfactuals is the "science" that researchers carry in their minds while modeling.

4.22.2020 7:41pm - (Replying to @Jabaluck @gelbach and 2 others) Now you are generalizing. I never insist on "my work", this is your hang up. I insist only on starting with cognitively meaningful and defensible encoding of what we know, combining it with data, and formally deriving what we wish to know. Has any of your model papers done that?

4.22.2020 7:13pm - Thanks for this positive and informative reviews. I found a short link here This is, if I am not mistaken, the first review of #Bookofwhy in a flagship scholarly journal. I hope to see more soon, so that readers of BOW will begin to feel mainstream.

4.22.2020 6:52pm - (Replying to @Jabaluck @gelbach and 2 others) Can we get to the causal part?. These papers are still in the data-fitting paradigm, taking "conditional ignorability" as a God given gift.

4.22.2020 6:37pm - Thanks for the feedback. And, now that you are at it, please read some of the antagonistic critics, as many as you can, to truly appreciate the stifling paradigms that #Bookofwhy has attempted to shift.

4.22.2020 6:28pm - (Replying to @Jabaluck @gelbach and 2 others) Tell me more on how economics has embraced machine learning, in substance please, not in hype. And please focus on that part of machine learning which is not curve-fitting, namely computer-aided statistics. FYI, CI folks consider CI to be part of ML, albeit above Rung-1

4.22.2020 6:22pm - (Replying to @Jabaluck @gelbach and 2 others) Beg to differ. The resistance generated was not to overselling novelty but to my insistence that new tools are needed. Challenge: show me one oversold novelty in, or in:

4.22.2020 12:47pm - (Replying to @gelbach @Jabaluck and 2 others) The stagnation we see in certain (unnamed) disciplines calls for courageous and rebellious spirit, not for professional humility (or timidity), which has led to the insular stagnation.

4.22.2020 12:36pm - (Replying to @jeffreywatumull) Yet, unless we suspect that subjects response was affected (positively or adversely) by the very idea of being selected for treatment, it is reasonable to take the un-selected subjects as control group.

4.22.2020 12:30pm - Kudos to Sacks et al for creating the CRAN program, which should be used together with their Biometrika paper - A comprehensive account on computing tight bounds for causal effects.

4.22.2020 12:28am - (Replying to @RemyLevin @HeerJeet and 2 others) I am not familiar with a lot of cross-disciplinary critiques, except the ones I have articulated here and here #Bookofwhy. Speaking of these critiques, I see concrete examples, formal proofs and powerful tools - not one shred of dishonesty or X-measuring!

4.21.2020 11:13pm - (Replying to @RemyLevin @HeerJeet and 2 others) Nice and rosy. Still, you would agree, I hope, that some fields, at some points in time, could benefit from some jolt of self reflection. And the jolt may not always come from inside, given the self-perpetuating structure of our academic culture.

4.21.2020 11:00pm - (Replying to @Jabaluck @autoregress and 2 others) Sorry, I did not realize that. So, back to waiting for a prominent economist to take a sober look at the education and culture dominating the field.

4.21.2020 7:11pm - Responding to critical yet constructive reviewers, we have revised our "generalizing experimental results.." paper:, and we are happy to report new discoveries concerning bounds and new Bayesian estimation of real data on Vitamin A supplement.

4.21.2020 4:21am - (Replying to @RWerpachowski and @Spinozasrose) I did not think you said it in a mean way. My point still is that, in order for the memory of the Holocaust to be long lasting we must tie it with revival and rebirth. And I do not think the directors of our 60 Museums understand it; you can hardly see a glimpse of Israel in any.

4.20.2020 4:18pm - (Replying to @RWerpachowski) My grandparents failed to escape from the train, and I know that they wanted to be remembered not by the train, but through their grandchildren lifting themselves from the ashes and carving a future of hope and dignity for themselves, their people, and their descendants.

4.20.2020 3:52pm - This picture, not the cattle trains nor the gates of Auschwitz, should decorate the entrance hall to each one of the 60 Holocaust Museums in the US.

4.20.2020 3:27pm - (Replying to @HeerJeet @guacamolebio and @beyerstein) I've never dreamed to hear this echoed in the middle of the "credibility revolution." #Bookofwhy

4.20.2020 4:21am - (Replying to @RWerpachowski and @Spinozasrose) I did not think you said it in a mean way. My point still is that, in order for the memory of the Holocaust to be long lasting we must tie it with revival and rebirth. And I do not think the directors of our 60 Museums understand it; you can hardly see a glimpse of Israel in any.

4.20.2020 10:12pm -

4.20.2020 4:21am - (Replying to @RWerpachowski and @Spinozasrose) I did not think you said it in a mean way. My point still is that, in order for the memory of the Holocaust to be long lasting we must tie it with revival and rebirth. And I do not think the directors of our 60 Museums understand it; you can hardly see a glimpse of Israel in any.

4.20.2020 10:12pm - (Replying to @VUspenskiy and @andrewheiss) Actually, the sequences of do-calculus rules have been systematized into an algorithm that gives us the result without going through the rules: But when I look at a diagram, I first ask if it contains one of the recognizable patterns stored in my mind.

4.20.2020 10:01pm - Naive questions are everyone's questions. Ans. The backdoor and frontdoor are logical consequences of do-calculus. So why do we decorate them with names? Because they are easily recognizable in the DAG, so we store them explicitly in our arsenal of tools, so skip re-deriving them

4.20.2020 4:53pm - (Replying to @EinatWilf) Does anyone understand why the American Jewish Committee would not acknowledge the dreams and ideas that inspired the Warsaw Ghetto Uprising?

4.20.2020 4:18pm - (Replying to @RWerpachowski) My grandparents failed to escape from the train, and I know that they wanted to be remembered not by the train, but through their grandchildren lifting themselves from the ashes and carving a future of hope and dignity for themselves, their people, and their descendants.

4.20.2020 3:52pm - This picture, not the cattle trains nor the gates of Auschwitz, should decorate the entrance hall to each one of the 60 Holocaust Museums in the US.

4.20.2020 2:25pm - (Replying to @danilobzdok @shakir_za and 3 others) Agree. It's an excellent Sunday read. But the message of this paper was "a reminder of how easy it is to fall into a web of paradoxical conclusions when relying solely on intuition, OR solely on statistics". We must snap out of statistics to get it right.

4.19.2020 2:15pm - Conditioning just gives us a more specific class of individuals, with all the problems of sparse data; it is still not the individual itself. This paradox highlights the difference, and the role of counterfactual reasoning in getting to the individual.

4.19.2020 12:41am - (Replying to @Physical_Prep) Not surprised that the #sportscience literature suffers from similar myths. ACE is merely one evidence for what we wish to know about an individual. See eg our recent blog on which individual is "in greater need" for a hospital bed.

4.19.2020 12:32am - (Replying to @omaclaren and @unsorsodicorda) I am genuinely trying to learn from the vast literature of DE. Can you give me the simplest example you have, time invariant, where the task of inferring effects can benefit from the teachings of DE? I've just given you one from which qualitative DE folks can benefit.

4.19.2020 12:12am - (Replying to @omaclaren and @unsorsodicorda) This sounds exactly like CI. In which case can DE folks get the causal effect of X on Y, given data taken from Pr(X,Y,Z) if the input is three unknown functions, x=f1(z,u1) y=f2(z,x,u2) z=f3(u2) and u1,u2,u3 are independent yet arbitrarily distributed random variables? How?

4.18.2020 11:57pm - My current interest in "personalized medicine" compelled me to read this new paper I must conclude though that, despite 3 decades of CI, "personalized nutritionists" still have to accept that there can be no "personalized" inference w/o counterfactuals.

4.18.2020 11:33pm - (Replying to @unsorsodicorda and @omaclaren) Can you summarize some of this literature in input-output language, as we folks do with CI? What must the input be like + what kind of data is taken and what kind of answers we get from the analysis?

4.18.2020 11:19pm - (Replying to @seema_econ) Yes, there sure is "a little virus going around"!

4.18.2020 7:14pm - (Replying to @omaclaren) I would be very happy to enlist the work done in CI as a modest contribution to the vast umbrella of DE, assuming the latter is open-minded. For example, the adjustment formula can easily be viewed as an exercise in qualitative DE, so is the Instrumental Inequality etc. etc. etc.

4.18.2020 4:09pm - Readers suggest that this tutorial video may well serve as an introduction to an online course in CI. Unfortunately, the videographer was sloppy on the slides, so I am posting the slides here: Feel free to use them in any class.

4.18.2020 3:30pm - I am hearing great passion for ODE, so I ask myself: "Why not use ODE for causal inference?" One answer: We don't have the functions that make up ODE. All we have are some qualitative properties, eg. who are the arguments of each function, and we still venture to estimate effects

4.18.2020 4:20am - Another exciting paper arriving at my desk reads: Causal Relational Learning: which promises to revolutionize causal inference the same way first-order predicate logic has transformed Boolean logic.

4.18.2020 3:36am - I am always happy to see "completeness" results in CI, but these results are especially gratifying, because graphical modeling of missing data problems ( have totally been ignored by MD practitioners. I hope things will change soon.

4.18.2020 3:04am - On the basis of the first 2 accessible sections, this is the best on line course on the market. Highly recommended. Especially for economists!!!

4.18.2020 1:17am - Here is a fun question that came up in conversation with my grandson. If it is bouncing off the moving piston that speeds up molecules as gas is compressed, how come slow-moving and fast-moving pistons result in the same temperature increase? A question for AI reasoning systems.

4.18.2020 1:04am - (Replying to @attilacsordas) What about "music breaks the silence". For every thing that is created the absence of that thing is destroyed.

4.18.2020 1:00am - (Replying to @shaul_ido) Trying to find one

4.18.2020 12:56am - (Replying to @mribeirodantas) I love it too. And this paper describes a machine that generate such data on demand.

4.18.2020 12:49am - (Replying to @firstmn59) Good for Rung 1 and Rung 2 (interventions), weak on Rung 3 (counterfactuals and SCM), so I would supplement it with chapter 4 of Primer:

4.18.2020 12:46am - (Replying to @juli_schuess and @Jean_DeCarli) Highly recommended.

4.18.2020 12:40am - Your question is a good one, and it reminds me to remind readers of this link which would provide them with a searchable file with all my past tweets (now numbered 46K). Please search for "cyclic" or "feedback".

4.17.2020 5:29am - I am retweeting your query to all our readers, I hope some can point you to a good on-line course based on Primer I lost track.

4.17.2020 3:19am - The more I listen to it, the more I feel I can speak Mandarin. Perhaps because cause and effect are universal or because (no, this is impossible!!) I can really understand Mandarin.

4.16.2020 10:13pm - Amazing photo. I believe it is the first public demonstration complying with COVID-19 "social distance" rules. Israelis protested yesterday government corruption and the erosion of democratic institutions.

4.16.2020 5:46am - Here is how Richard Feynman explains why molecules speed up when gas is compressed. About 5-6 minutes into this video he explains how a gas molecule, fairly stupid one, knows that it must increase it kinetic energy to make up for the work done on the gas.

4.16.2020 5:12am - Replying to @omaclaren) The choice of the word "governing" may be misleading. He meant the law is never violated, which does not imply that it is sufficient for, or that it cognitively "explains" all phenomena.

4.16.2020 4:42am - This is incredible! Straight in the Journal of Statistics Education?! If it's true that "The Book of Why (Pearl and Mackenzie, 2018) increased the motivation to teach concepts that Pearl calls the causal revolution" then writing the book was worth every drop of ink (key stroke.)

4.16.2020 4:18am - Conservation laws do not satisfy our quest for causal explanation. Does a molecule know that it needs to conserve energy? So what makes it speed up in a Venturi pipe? Or when gas is compressed? Feynman always preferred putting himself "in the shoes" of those molecules. Me too.

4.15.2020 7:05pm - (Replying to @weissiam) A theory that explains everything, from quantum mechanics to cosmology, humbles the wise to remain silent ("Yidom" Amos, 5). I must remain silent till I learn how to use this theory to resolve Simpson's paradox, or to spot imperfect experiments. Will be back if wised.

4.15.2020 3:25am - Very pleased to find a powerful completeness result in the theory of transportability: This time the heterogeneous data sets can have missing components and the targets can be group-specific causal effect.

4.15.2020 1:05am - (Replying to @rodakker) Thanks for the link. Again, the abstract raises my objection. The link is MORE than just interaction, its confounding too. Traditional mediation analysis attempts to cast a causal problem in statistical language - an impossibility, as shown here:

4.14.2020 10:40pm - A pay-wall prevents me from reading this whole paper But the abstract raises a huge red flag: "In psychology, the causal process between 2 variables can be studied with statistical mediation analysis." IMHO, there is no such thing as "Stat. Med. Analysis."

4.14.2020 9:04pm - Happy birthday, Tel Aviv I've never seen your beaches so sad Your sons and daughters alone, yet playful and glad, Your true face, unmasked, smiling openly, wide Chosen by tomorrow, a city in white

4.14.2020 4:37am - Malki and Danny, my two fallen souls, Will the darkness of the night remember Two shooting stars went by? Shining their most brilliant lights, their noblest, In their last dance through the sky?

4.14.2020 12:10am - (Replying to @aminsaadou) Of course, provided the editor of Econometrica commits to publishing it. But why do we need a paper? Readers and students will benefit more from seeing economic solutions to the questions I posed to Heckman here: A toy problem is worth 100 papers.

4.13.2020 7:43pm - Econonists are trained to find flaws in models, however plausible, so much so that they have lost the capacity to ask: "Suppose the model was correct, what do I do with it?" As a result, they lost the skill to repair the flaws they labor to discover -- others (epi?) are doing it.

4.13.2020 5:59pm - (Replying to @stephensenn @pash22 and 2 others) Good try, but causal calculus was never proposed to represent "what we mean by causal effects" as did RCTs. "Causal chains" (eg DAGs) and "counterfactuals" were proposed as the building blocks of causal thinking. And they are!

4.13.2020 5:56am - (Replying to @thosjleeper) Oh, thanks, I missed the beginning of the chatter; got there only when the question was put for a vote: who is better? Eager to see the initial trigger.

4.13.2020 3:16am - Funny, we asked this question six years ago: and received 38 juicy comments. The summary then describes sympathetically what it feels like being an economist, denied the guidance of graphs, and pretend you don't need any.

4.13.2020 1:28am - Anyone concerned with inferring causes from effects as well as predicting individual behavior from group data has probably stumbled on the questions raised in this blog entry, and would surely benefit from the answers provided by Carlos Cinelli.

4.12.2020 2:11pm - We are grateful to Congressmen Adam Schiff and Steve Chabot for taking a strong, bipartisan stand on behalf of justice and press freedom . @justicefordanny

4.12.2020 2:32am - "Walking among alligators" means seducing scientific leaders to undergo a paradigm shift under hypnosis, thinking they knew it all along. No one will forgive he/she who finds your wallet and proves that the time you spent under the lamppost - an entire career - was a waste.

4.12.2020 1:55am - (Replying to @phyecon1) I was not aware of this incident. My Pakistani friends always depict Jinnah as a champion of modernity and moderation.

4.11.2020 6:12pm - (Replying to @quantadan) I think I owe this affliction to my Kibbutz, in upper Galilee, for sending me (1954) to study choir conducting in Haifa, see, and to my son Danny, who was the copy-editor of my first book, and kept on complaining about my writing: "It doesn't sing, Dad!"

4.11.2020 5:06pm - (Replying to @vthorrf) Brilliant! I wish he lived in the digital age. He would have become an AI pioneer; he wouldn't have allowed philosophy to linger in philosophy departments.

4.11.2020 2:30pm - (Replying to @vthorrf) Is it Hume, or Locke?

4.11.2020 4:20am - (Replying to @ClaudeAGarcia) At the risk of provoking more anger I would just say that, understandably, the naked is angry at the sunlight, though we are all naked under our pajamas, and we all know it.

4.11.2020 12:31am - (Replying to @yakir_bella) If you can find a translator, I would waive my royalties and donate a free copy to every college library. Why? Because Israel has limited resources, and can't afford to see its science education taken over by the data-fitting style of machine learning - it's addictive & blinding.

4.11.2020 12:16am - If we look carefully at his hand movement, we see that the little one is testing the "mediation formula"; how the spiral wheel turns and stops with and without the linkage in between. He is surely climbing rung-3, toward creative scientific thinking.

4.10.2020 3:56pm - (Replying to @11kilobytes) Two reasons for anger: (1) "We have been doing it already in 19xx" (2) "There is more to it than #Bookofwhy"

4.10.2020 3:16pm - I find this tweet impossible to disagree with, though I am suffering from an illusion that you won't find a scientist who hasn't read #Bookofwhy, barring those who are angered by it.

4.10.2020 2:29am - (Replying to @Michael_D_Moor) Hillarious! No wonder economists refuse to solve "toy problems" -- everyone would be able to see how their wheels turn.

4.9.2020 8:16pm - (Replying to @tylerlu) And the chemist will say: Why study connectionism if we can simulate the chemistry of proteins and have machines solve the human brain mystery.

4.9.2020 7:37pm - I have discovered a spec of truth in Bajwa's fable: The order to keep Shaikh in custody on the grounds of ‘public safety’ was taken INDEED under pressure from external agencies -- I was one of those agencies when I reminded the world who Shaikh is and what he did to my son Danny.

4.9.2020 6:27pm - This is my favorite photo of Danny . The article echos our hopes for justice, moderation and press freedom.

4.9.2020 3:42am - I was about to send warm wishes to all readers celebrating Passover, when this came in. Obama is a much better orator, so I'll ride his words and second his wishes: Happy Passover !!!

4.9.2020 3:31am - Only those who have travelled this road in normal times would appreciate what it takes to bring it to this state of Covid-19 paralysis. It is the road connecting Haifa and Tel-Aviv on Passover eve.

4.8.2020 7:35pm - You go with Sarsour and Omar, Senator, not me. You have brought shame and immeasurable harm to your (former) people. Tonight, as Jews say: "Keivan Sh'Hotsi et Atsmo Min Haklal" ("estranged himself to the community") we will be thinking of you, among other painful disappointments.

4.8.2020 1:48pm - Anyone who wishes to obtain the solution manual for the beautiful examples in should write to and indicate that it will be used for self-study and will not undermine instructors who assign these questions as class homework #Bookofwhy

4.8.2020 2:51am - (Replying to @y2silence) Best compliment an author can dream of. #Bookofwhy

4.8.2020 1:09am - (1/ ) (Replying to @bariweiss) My memorable Seder was on April 23, 1948, 3 weeks before Israel was created and attacked. We, kids, heard all the threats of "monumental genocide" and "rivers of blood" that our neighbors broadcast from Cairo and Beirut, and were curious to see if our "adults" would dare
4.8.2020 1:16am - (2/ ) (Replying to @yudapearl and @bariweiss) celebrate Passover under such conditions. My grandfather just read the Haggada, as usual, got to the story of the five Rabbis who defied Roman's rule and chanted the exodus in Bnai-Brak (my home town), stopped, smiled, looked us in the eye and said: "So, what's new?"

4.7.2020 11:12pm - (Replying to @fuzzydunlop123 @omaclaren and 7 others) Thanks for posting. I was not aware of this insightful paper on the connection between the two models.

4.7.2020 10:45pm - For the life of me, why should hypocrisy and stupidity evoke more twitter chat than being a Zionophobic racist? Who said inconsistency is a greater perversion than immorality.

4.7.2020 7:03pm - Another juicy proverb, just provoked by a discussion with a "missing data" expert: "When it comes to perplexity, nothing tops the science of inertia, except perhaps the inertia of science."

4.7.2020 5:36pm - Any reason you did not like the ending?: "Data science is not a mirror through which data look at themselves from different angles under different makeups"

4.7.2020 4:02am - (Replying to @Razorwindsg) We do not have an organized movement per se. But we have a commitment to education (on-line) and a strong conviction that commonsense will prevail. I would start with spreading the intellectual content of CI in whatever circles you dance; most circles would benefit from it.

4.7.2020 3:22am - A more effective way to write Pakistani Prime Minister Imran Khan in support of justice for Daniel Pearl would be to use these three email addresses: Cc: Thanks, and may justice prevail.

4.7.2020 2:16am - (Replying to @nickchk @Jacobb_Douglas and @Fhanksalot) Discussants on this thread might enjoy a glimpse at the general condition for Z to be a valid IV. The condition goes: 1) There is an unblocked path between Z and X, and 2) Every unblocked path between Z and Y contains an arrow into X. This condition confirms of course the cases

4.6.2020 3:08pm - Responding to readers comments, we have improved our discussion of "Which Patients are in Greater Need?..with reflections on COVID-19" Same with the related post, that argues for "data-interpreting technology"

4.6.2020 3:56am - (Replying to @maliniw90th) Your father is lucky. I wish my daughter would ask me to explain Fourier transform or integration by parts. Fun topics.

4.6.2020 1:55am - Speaking of DL and causal inference (CI) and keeping with our commitment to on-line education, I am retweeting here a video-ed lecture on the subject It's more than a year old, but covers many of the questions raised here, squarely and transparently.

4.5.2020 1:36am - (Replying to @desai_pratik) I haven't kept ups with Soar type of projects. From what I remember this was a "production system" and we, expert systems folks, moved from rules to causal models. But the cog. science people still use Causal Bayesian Networks as a model of cognition and "understanding".

4.5.2020 9:05pm - Agree. Yoshua Benjio is one of the few DL leaders who understands the role of causality in AI, the rest are either avoiding the issue or paying curve-fitting lip-service to it. We, AI and society, will be paying dearly for this neglect, as I note here:

4.5.2020 8:44pm - (Replying to @VincentAB @causalinf and 2 others) I agree of course with your conclusions. What I do not understand is your self-doubt. There is only one definition of IV and one way of verifying it. Only economists are confused today about IV's because, having vowed to shun graphs, they are still debating what "exogeneity" is.

4.5.2020 8:29pm - (Replying to @adepstein1 @RLong_Bailey and @Keir_Starmer) And I hope UK Jews get the courage to talk honestly to @Keir_Starmer and tell him what his litmus test is: How he talks to Labor members about Israel and Zionism and lasting peace in ME, not how he flatters UK Jews.

4.5.2020 3:16am - Readers who asked if they can help, please write to @ImranKhanPTI , the prime minister of Pakistan, and express your hopes to see his government appeal the acquittal ruling, thus reaffirming Pakistan's commitment to universal values of justice and the sanctity of human life.

4.5.2020 2:55am - (Replying to @RonKenett and @mario_angst_sci) I tend to side with @mario_angst_sci on this issue. There is a profound difference between a tool that makes you ask: "How come no one told me?" and one that leaves you blank even after being told. The former lets you do things you always wanted to do and couldn't!

4.4.2020 11:14pm - (Replying to @DocGTLBrown and @mario_angst_sci) Extremely pleased and, if anyone was busy preparing a "guide for students seeking enlightenment" I would recommend adding your course to the White List. One needed ingredient: Add CI to the title, to show that your department is not bowing to pressures or taboos.

4.4.2020 5:54pm - My thoughts on Imben's ideology are here: Indeed, the econometric leadership will have to answer some tough questions to the court of history. Econ. students will be summoned as witnesses, and econ. textbooks as material evidence. Wish I live to see it!

4.4.2020 3:28pm - Likewise, I've been advising schools that advertize for students, postdocs and faculty to state explicitly their interest in CI education and research, else they would not get the students and faculty they are looking for. "Times they are achangin" (Dylan 1964).

4.4.2020 3:14pm - This past week, the Stanford's HAI Institute has organized a virtual conference on "AI and COVID-19," a video of which is now available: I have asked the organizers to share the following note with the participants:

4.4.2020 2:41pm - And how on earth is it that prominent leaders and educators in respectable scientific disciplines do not ask themselves the same question to lift their fields from the margins of the causal revolution? Historians of 21st century science will ask this question.

4.4.2020 7:21am - To all readers who shared hopeful thoughts with us, we are grateful and happy to inform you that the government of Sindh has ordered that Daniel's murder suspects will be kept in detention for another 90 days, pending an appeal. Thanks for being with us.

4.2.2020 8:13pm - A new blog-page, carrying a humble contribution of causal inference to the fight against Covid-19 was just posted here: It asks: "Which Patients are in Greater Need?" and displays the options in vivid colors.

4.2.2020 5:27am - It is a mockery of justice. Anyone with a minimal sense of right and wrong now expects Faiz Shah, prosecutor general of Sindh to do his duty and appeal this reprehensible decision to the Supreme Court of Pakistan.

4.2.2020 3:21am - (Replying to @Abel_TorresM @dileeplearning and 3 others) Agree on first, disagree on second. Structural Causal Models exhibits true understanding of a domain (defined by the variables in the model) in that it answers all causal and counterfactual questions about that domain, as shown in #Bookofwhy chapter 1, "The Mini-Turing Test."

4.1.2020 4:19pm - (Replying to @Dr_Cuspy @tdietterich and @MarcioMinicz) I would hate to use the word "autonomously" in a definition, b/c it is hard to tell if a program does things by choice, agency, or by being programmed to do it. "Predicting consequences" is easier, we can just ask the program to predict, and test the answer against outcomes.

4.1.2020 3:57pm - (Replying to @HundredthIdiot and @deaneckles) Really? I was told many readers filter their tweets by subject matter, and will get only Tweets containing those hash tags. If it ain't true, I'll be more than happy to gain 10 precious characters not using #Bookofwhy (I just did!!).

4.1.2020 3:47pm - (1/ ) In re-reading Holland (1986) I noted that he was the first to define CI as a missing data problem, through his "Fundamental Problem of Causal Inference", which so many PO folks love to quote. Today, in contrast, we are classifying "missing data" as a causal inference problem
4.1.2020 3:47pm - (2/ ) (see for example Wow! How "the times they are a changin" (Bob Dylan, 1964). #Bookofwhy

4.1.2020 3:20pm - For readers who are re-raising @deaneckles question, I am retweeting my reply (below) and add that I also praise DAGs for reminding us that it is not the end of the world, and guiding us to augment our knowledge to get identification: IV, front-door, do-cal. etc. #Bookofwhy

4.1.2020 1:37am - Glad we have a causal model for COVID19 testing, so that we can accumulate facts and judgement into one arena, and talk one language. #Bookofwhy

4.1.2020 1:23am - Thanks for sweet nostalgia. I often play with the idea of writing: "Statistics and CI - 30 yrs later", but not sure JASA would publish it. Holland's paper has shaped the mindset of most living statisticians working on CI, some irreversibly.#Bookofwhy

3.31.2020 8:43pm - Missing data is when Chinese journalists disappear, eg and statisticians later fix with multiple imputation. #Bookofwhy

3.31.2020 7:15pm - For coordinating, comparing and making sense of pandemic data sources, corrupted by local idiosyncrasies we need causal meta-analysis. The theory is available: and is awaiting an elite force of CI PhD's to teach & train practitioners. @EpiEllie #Bookofwhy

3.31.2020 4:56pm - This incredible scene comes straight from the twilight zone, where "The wolf lives with the lamb, the leopard lies down with the goat, the calf and the lion and the yearling together; and little child leads them." (Isaia 11:6). No child in sight, unfortunately.

3.30.20 11:28pm - The warm reception of my conversations with N Jewell tempts me to post another ancient video: It took place 10 years ago, on the foundations of causal reasoning; a topic more alive now than ever, especially while fighting for explanations #Bookofwhy

3.30.20 8:04pm - (Replying to @MEMRIReports and @HananyaNaftali) I know a few US Universities who would vie for his scholarship in the department of Mid East Studies.

3.30.20 3:25pm - The crucial methodological part of the debate rests with fusing coherently all the data coming from so many diverse and noisy sources. We have a calculus+algorithms for doing it, awaiting experts trained in CI and listen; see #Bookofwhy

3.30.20 3:07pm - I'm flattered, but don't stop here. Continue to Primer, and join the common sense revolution - every rebel counts. #Bookofwhy

3.30.20 12:09am - The chloroquine debate should be settled by domain experts. As a methodologist, I can only state that algorithms have been developed that can look at data and estimate the probability that a drug is harmful to a given individual. I assume the experts are reading #Bookofwhy

3.29.20 6:00pm - Written especially for revolutionary-minded people like you. Who said epi is not poetry? #Bookofwhy

3.29.20 3:40am - Another interesting paper: which provides an insightful analysis of cross-world assumptions in mediation analysis. See #Bookofwhy chap. 9, and for nice plots of the mediation formula in linear, logistic, and probit models #Bookofwhy

3.29.20 3:08am - Retweeting an interesting paper by a Microsoft team on one of the most demanding task performed by humans: system debugging It combines causal and counterfactual logic, as opposed standard debugging softwares which are statistically driven. #Bookofway.

3.29.20 2:17am - (Replying to @mgaldino) This is only part of the answer. Let's not forget that formalization serves both: automation and coherence. Applied researchers, following their informal intuition might find themselves producing contradictory recommendations (as in Simpson's paradox) #Bookofwhy

3.28.20 9:17pm - (Replying to @awmercer) Thanks for your honest and informative answers. Recall however that for us, AI-ers, "makes the most sense" is just the beginning of a new challenge: How do we represent (on a machine) the knowledge that tells you that one framework "makes more sense" than another. #Bookofwhy

3.28.20 6:08pm - (1/ ) (Replying to @awmercer) This is the first time I hear of structural equations in the service of the survey field, which makes me super curious: Do these equations enjoy different notation than regressions? Different treatment? Surely, the model-free approach misses something; it misses an opportunity
3.28.20 6:16pm - (2/ ) (Replying to @yudapearl and @awmercer) to benefit from valuable information coming from the model, when such is available to the analyst. What is the simplest survey example in which we can see the tradeoffs between the two approaches? A toy example is worth tons of scholarly words. #Bookofwhy

3.28.20 4:08pm - Another thought evoked by your informative description of survey methods. Since these methods rely on the "no-confounding" assumption, how did survey experts articulate this assumption mathematically? Or they just carried it in the head? What's in the tech literature? #Bookofwhy

3.28.20 3:44pm - (Replying to @shirokuriwaki and @deaneckles) This paper is not very helpful because, following the PO tradition, it assumes away the hard part of the problem. Quoting: "The conventional solution to this problem is to assume ignorable treatment assignment and overlap (Rosenbaum and Rubin 1983)" #Bookofwhy

3.28.20 3:41am - (1/ ) I'm glad, Sean, that our brief exchange has resulted in your great clarification of the issues, from which I have learned a lot. Two thoughts come immediately to mind: (1) It is a blessing that we can enjoy a division of labor between CI and statistics, the former generates
3.28.20 3:41am - (2/ ) causal estimands, the latter estimate them. Note though that the former is not totally oblivious to the type of data available. Different types of data will result in different estimands. eg.,experimental vs. observational, corrupted by missingness or by proxies or by
3.28.20 3:41am - (3/ ) differential selection etc. (2) I don't buy the mystification of "collecting adequate data". I am in the business of automating a scientist, so, if there is human judgement involved in the data collection task, we do not stop here and surrender it to humans. We see it as an
3.28.20 3:41am - (4/ ) invitation to ask: what knowledge allows you to decide that some data are "more adequate" than others. We then model that knowledge and automate the process. I strongly suspect that behind this piece of knowledge you'll find a causal model. I am willing to bet!! #Bookofwhy

3.28.20 3:29pm - Truly appreciate your substantive reply. So, generalization across population is the common theme here. In we solved such problems using graphs. Have we missed something by not listening to the survey literature and to the tools developed there?#Bookofwhy

3.28.20 3:19pm - (Replying to @awmercer and @deaneckles) Great. Now suppose we have good understanding of confounders. What formal language is available to us to articulate our understanding, so that we can process it coherently when the number of confounders exceeds our mental capacity? Is Meng's language sufficient? #Bookofwhy

3.28.20 3:11pm - It is a great paper, and I retweeted it in Dec. 2020 with a warm blessing. However, I am still in a learning mood: Survey methods have been around for 2 centuries. What tools have been developed that we, CI folks, can use to speed up our research agenda? #bookofwhy

3.28.20 2:46pm - (Replying to @agostbiro and @deaneckles) No one doubt it. The entire success of DL rests on this surprise. But we, CI students, have certain problems on our plate for which we cannot find (yet) salvation in understanding correlation alone. And we are begging for help. #Bookofwhy

3.28.20 1:39pm - I am retweeting with the hope that perhaps some other readers beside @deaneckles can help us on this question, which has come up again and again. In a way, ML/DL folks are operating as survey samplers, so they too could help us: What can we learn from your experience? #Bookofwhy

3.28.20 12:45pm - There is another point to this pseudo-critique. Suppose you launch a very smart discovery algorithm and get a causal structure, what then? If you do not teach your students how to leverage it to answer causal questions we are back to where model-free analysts are today.#Bookofwhy

3.28.20 12:29pm - (Replying to @ewerlopes and @Muenchner_Junge) Thanks for posting this course. I have seen many advertised under the catchy title "causal inference" which are doing ONLY estimation, leaving the assumptions to divine intervention. This course has Primer in its reading list - a faithful sign of enlightenment. #Bookofwhy

3.28.20 12:11pm - (Replying to @AngeloDalli) There could be some other type of knowledge guiding statisticians in what they call "design". But the fact that they treat it as divine wisdom prevents us, AI researchers, from automating it. We will, eventually. #Bookofwhy

3.27.20 8:32pm - (1/ ) I beg to differ. For readers interested in my unbiased opinion, this review is ill-informed, ill-motivated and misleading. Bent on showing that "there is more to it than #Bookofwhy", sweating hard and unsuccessfully to find any such "more to it", the reviewers neglect to tell
3.27.20 8:32pm - (2/ ) readers what they would miss not reading the book. The quote: "causal inquiry cannot be reduced to a mathematical exercise nor automatized." is truly indicative of the occult "more to it" agenda of the review, and its failure to appreciate how much has been automated already.
3.27.20 8:32pm - (3/ ) Another unbiased opinion I have regarding "the particular value of randomized experiments". I challenge anyone to show me a clearer/deeper exposition of RCTs and their "particular value" than that given in #Bookofwhy ch. 4, The Skillful Interrogation of Nature, Why RCTs Work?

3.27.20 7:53pm - (Replying to @blake_camp_1 @kchonyc and 2 others) I am late on the philosophy of SSL, so I commend you for giving us a glimpse of what it does. Can you elaborate on how fillings blanks can get you a causal effect without some causal assumptions encoded somewhere in the system? #Bookofwhy

3.27.20 7:42pm - (Replying to @omaclaren and @analisereal) In which case these undergrads should be able to prove that back-door condition is sufficient for identifiability in a simpler, static system, with no derivatives. They should also be able to run DAISY and decide yes/no on any of the models described in #Bookofwhy. Eager!

3.27.20 7:27pm - (Replying to @omaclaren) I would qualify it a bit. ‘Modernity’ is the sum total of things that have been accomplished in the past 3 decades regardless of any particular formalism. This is how #Bookofwhy defines the "causal revolution", and it lists those things explicitly, task after task.

3.27.20 7:16pm - (Replying to @analisereal and @omaclaren) I would add that structural models are causal entities and should be annotated by arrows Y<--f(X,e), not equality signs. Once the distinction is made explicitly, I embrace the paper as "relevant", else, "suspect of generating confusion" as in #Bookofwhy

3.27.20 2:51pm - (1/ ) (Replying to @seanjtaylor) "Useful" is a rich concept. Statistics papers, formal logic, cognitive science, even history books and Greek mythology have been useful to my work. But when it comes to causal questions, a paper that does not articulate the question in mathematically tells me a lot about what
3.27.20 3:19pm - (2/ ) (Replying to @yudapearl and @seanjtaylor) to expect from reading it. It tells me that either (1) the paper deals with stat. estimation after assuming away all causal considerations, or (2) it has nothing to do with causation despite the catchy title, or (3) the author is confused - like many regression analysts -
3.27.20 3:19pm - (3/ ) (Replying to @yudapearl and @seanjtaylor) writing regression and thinking causation. It does not make the paper useless, but it warns you that it ain't going to be easy to excavate for the relevant material if the has any. So, if your time is valuable, you already saved 1000 hours, all for want of notation. #Bookofwhy

3.27.20 1:49pm - (Replying to @blake_camp_1 @tyrell_turing and @ylecun) No need to toy. Just look at the input information. If it contains a causal element (or interventional data) then your toying may succeed, if not, you can't create it from data alone. It's like the conservation of energy (Helmholtz, 1847.) #Bookofwhy

3.27.20 1:42pm - (1/ ) You almost got it correctly. You can read 1000 papers, for 2000 reasons, but if you want to know ahead of time whether it deals with causal questions formally, then it is not a matter of "preference" or "taste", it is a matter of distinguishing causal from statistical
3.27.20 1:42pm - (2/ ) quantities and, then, the use of do(x), Y(x), or DAGs is a good indicator whether a distinction was attempted. I've saved literally thousands of hours by this filter and I would be curious to know of any causally-relevant paper that I've missed. Notation reveals! #Bookofwhy

3.27.20 1:56am - (Replying to @tobytfriend @davidpapineau and 3 others) Good catch, thanks. There are indeed 3 notational indicators to causally-minded articles: do(x), Y_x, or good old fashion DAGs. Thanks. #Bookofwhy

3.27.20 11:44pm - (Replying to @ewerlopes) It is very kind of you to trust me for giving such a course. I'll do some checking around and see if it can be realized given everyone's constraints. #Bookofwhy

3.27.20 1:45am - Responding to interests in the foundations of causal inference, and lacking resources/army/time to marshal a full Zoom course, I am posting a video interview with N Jewell: Introduction to Causality, Part I,, Part II,, #Bookofwhy

3.26.20 7:33pm - (Replying to @btolgao @tvladeck and 3 others) I've been in this playground since 1990, see The PC algorithm (Verma's IC) may discover beautiful structures and we know its limitations. Jona's-type algorithms rest on non-scrutinizable assumptions (eg. additive noise), may turn out promising.#Bookofwhy

3.26.20 4:16pm - (Replying to @btolgao @tvladeck and 3 others) When the DAG is not known, it means the knowledge is not available; we need to acquire that knowledge by experimentation guided by analogies and metaphors, like the Greeks did. Not like the Babylonians. #Bookofwhy

3.26.20 4:09pm - (Replying to @Jabaluck @VincentAB and 2 others) No, I have not seen those "other approaches" arrive at those insights. Let's take the simplest of all: backdoor. Can you illuminate us on how a DAG-less "approach" would arrive there. But, please, let's use input-output, no ignorability acrobatics. #Bookofwhy

3.26.20 3:58pm - (Replying to @deaneckles @tvladeck and 2 others) Yes, we often do not have the knowledge required to answer our questions - I praise DAGs for telling us. We often have 3 equations with 4 unknowns - I praise algebra for telling me that the solution is not unique. Who else would tell us? Statistics? PO? #Bookofwhy

3.26.20 3:50pm - (Replying to @peterlorentzen @ladder2babel and 3 others) ?????????????????

3.26.20 2:35pm - (Replying to @DanielHGill @davidpapineau and 3 others) No, orthogonal is you have left from an orthodentist who failed dentistry school. Same as what's left from big-data lacking a causal model. #Bookofwhy

3.26.20 2:14pm - (Replying to @davidpapineau @totteh and 2 others) The nice thing about modern thinking is that you can tell orthogonality w/o even reading the paper. Just look at the equations, if all they have is probabilistic expressions, no do(x) and no counterfactual subscripts - forget it, its traditional statistics, no causation#Bookofwhy

3.26.20 1:46pm - (Replying to @totteh @davidpapineau and 2 others) Beg to disagree. Meng's paper is orthogonal to the issue of randomization. It is written in the best tradition of statistical orthodoxy - avoid the C-word, and avoid causal thinking. #Bookofwhy

3.26.20 1:21pm - (1/ ) (Replying to @deaneckles @rabois and @Jabaluck) You have assumed correctly. The #Bookofwhy is all about the insufficiency of "big data" to get us over from association (rung 1) to causation (rung 2 and 3), no matter how skillful we are in processing the data. Large n only gets us from samples to distributions. Another push
3.26.20 1:32pm - (2/ ) (Replying to @yudapearl @deaneckles and 2 others) is needed to get us from distributions to causal conclusions. The push may come from interventional studies, or from causal models. Note that RCT's must invoke interventions -- randomization alone is as useless as large n.

3.26.20 1:17am - (Replying to @Jabaluck @VincentAB and 2 others) The best way to address such complaints is to ask: Do you acknowledge the existence of some inference tasks which DAGs perform well, and which the alternatives find intractable, that is, close to impossible? #Bookofwhy

3.25.20 9:22pm - (Replying to @peterlorentzen @rabois and 2 others) Sorry, I haven't been around to catch what the dispute is about, or what @rabois said that triggered @jabaluck anger. I am not sure #Bookofwhy can help but I have found it a good start for any discussion about cause and effect.

3.25.20 1:27pm - I'll be back!!! I promise!!!

3.25.20 1:03pm - This is the first Certificat I have seen in "Causal Data Science"! Congratulations!!! I am anxious to see the first PhD Diploma in Causal Data Science, perhaps 4 years from now. #Bookofwhy

3.25.20 6:34am - (1/ ) I used to think that a journal editor can help refresh a field by inviting "survey articles" from outsiders. I no longer think so, b/c the questions that controvertial papers evoke need be answered by direct interaction with the author. My recommendation: Publish a dialogue
3.25.20 6:34am - (2/2) (a technical one) between the outsider and the top leader in the field that seeks fresh air. Anecdotally, I have just finished such a dialogue (for a prominent journal) and the editor who appointed the interviewer rejected the outcome. Lesson: that's what's needed #Bookofwhy.

3.25.20 5:34am - (1/ ) Feldman's observations are insightful and thought provoking. I think we, scientists, should learn from politicians (don't choke!). They knew that power corrupts and instituted 3 branches of Gvt to check on each other. We do not have a "checking" branch to tell any of our
3.25.20 5:34am - (2/ ) self-sustaining "fields": Hey we are X years behind the time! We are waiting for outsiders to do it, which is too slow a process. Here is a counterfactual thought: Vice Presidents in charge of "Fresh Air", that is, innovation, openness, cross-field communication etc.#Bookofwhy

3.24.20 10:44pm - (1/ ) To assess where quantitative psychologists stand on the road up Mt. Causation, I examined the citations in the "taboo paper" and noticed a dearth of causally sound papers in psych.. [And I thought economists have the hardest time with modernity.] While
3.24.20 10:44pm - (2/ ) the survey paper by Foster is pretty good (for 2010), not much else from a field hopelessly sunk in SEM culture. I knew that Psychometrika and JSEM can't shake stagnation, but I thought the rest of the field is more progressive. Cognitive Psychology,
3.24.20 10:44pm - (3/ ) for example, is a model of progressive thinking. It would be helpful to hear from readers active in this field. Am I off in this gloomy assessment of quantitative psychology? #Bookofwhy

3.24.20 6:36pm - I have an even shorter link and note that the authors have not yet used the heavy guns against the supremacy of RCT's: Selection bias (as discussed here: ) #Bookofwhy

3.24.20 6:04pm - I hope this link works:

3.24.20 5:56pm - The Taboo Against Explicit Causal Inference in Nonexperimental Psychology is undergoing a harsh examination here file:///C:/Users/Judea/Downloads/Grosz%20et%20al.%20(2020,%20preprint)%20The%20Taboo%20Against%20Explicit%20Causal%20Inference%20in%20Nonexperimental%20Psychology.pdf

3.23.20 10:36pm - (Replying to @AdanZBecerra1 @mayfer and @_MiguelHernan) Of course. Likewise, doing real world dpi is "doubly-super-hared," if not impossible, without SCM. I have not heard about "other causal frameworks" that distinguish "cause of death" from "observed and death" and estimate the former, not the latter. #Bookofwhy

3.23.20 10:24pm - (Replying to @AdanZBecerra1 @mayfer and @_MiguelHernan) In the case of CV, it is reasonable to assume: "CV caused death" = "CV verified & death." Counterfactual logic is needed when we doubt this identity. Note that "verified" is still different from "tested positive" which is behind the statistics we are getting.#Bookofwhy

3.23.20 4:27pm - (Replying to @djnavarro) My personal apology to you, Danielle. Sorry.

3.23.20 3:02pm - (Replying to @AshbyHarper) Had I been in agreement with the ideology of the writer, I would not have apologized. But I object to everything he stands for, so I apologize for having created an unintended false impression. Sorry!

3.23.20 5:26am - (Replying to @Satoohn and @ChrisAdamsEcon) The weather predictor is observing two different needles, goes through some mental process and makes a prediction. Someone asks "what is the effect of Needle-1 on the prediction, given data from the past 100 days. This is where collinearity hurts. #Bookofwhy

3.23.20 3:38am - (Replying to @athos_damiani and @_rchaves_) I give publicity to many articles that invite humor, criticism and, God forbid, even ridicule.

3.23.20 3:22am - (Replying to @athos_damiani and @_rchaves_) Sorry, the title was 2+2=4. The paper may have jumped to 2+2=5, but the title is innocent. BTW, Who is God?

3.23.20 2:31am - (Replying to @DVBartram) Reconsidered. Retracting. Havn't foreseen the side effects or the implications. Asking for forgiveness. Which, btw, is a counterfactual notion demanding formal explication. But I'd settle for ordinary, pre-scientific forgiveness.

3.23.20 1:17am - (Replying to @_rchaves_) What's wtf? What's transphobic? What have I done wrong this time? Can't we enjoy a title saying 2+2=4 w/o offending a mathematician (or an economist)?

3.22.20 12:48pm - I heed to your proposal and offer our Crash Course to all data-intensive professions: I mentioned economists only because they coined the name "bad control" and have been most obstinate in refusing the science of its remedies. #Bookofwhy

3.22.20 12:25am - I see that you singled out "psychologists" as potential beneficiaries of our "Crash Course on Good and Bad Controls". What circles in psychology deserve this honor? #Bookofwhy

3.21.20 2:45pm - (Replying to @hangingnoodles and @aminsaadou) Causal modeling tools beat and escape the Lars quote better than unaided "thinking", which some (eg., Keynes) see as a superior tool to rescue science and policy makers from the complexities of social systems. #Bookofwhy

3.21.20 2:31pm - (Replying to @MehboobFarah) The distinction between "good and bad controls" is not used for "constructing models" but for "utilizing models" once constructed and deemed plausible. #Bookofwhy

3.21.20 12:55pm - (Replying to @aminsaadou and @hangingnoodles) The causation tools available nowadays are EASY for everyone (with the exception of some outdated old-guards), see, and they also make clear when old tools are adequate. #Bookofwhy

3.21.20 12:36pm - For the many students of economics who express frustration with the way a problem known as “bad control” is evaded, if not mishandled in econometrics, our "Crash Course" in now accessible as Technical Report: Enjoy and teach your professor. #Bookofwhy

3.21.20 5:10am - Proud to congratulate my colleague-in-law, professor Furstenberg, on winning the Abel Prize in Mathematics: Wishing El-Al Airline a quick recovery, so he can fly to Norway to receive the Prize in person. #Bookofwhy

3.21.20 1:28am - "Causal Transfer" and "Imitation Learning" is what called my attention to this new paper:, plus the fact that it approaches the problem scientifically, as opposed to data-fittingly. A prerequisite to data-science. #Bookofwhy

3.21.20 12:26am - To clarify, my comment below relates to @LarsPSyll statement "Econometric modelling should never be a substitute for thinking." which some take as a license to substitute modelling with "thinking". Modeling is a necessary amplification of the solid part of our thinking #Bookofwhy

3.20.20 7:31am - (Replying to @hangingnoodles) I am willing to accept the unmanageable nature of social systems. What I find hard to accept is that an unaided and fallible machine called "commonsense economics" is better equipped to manage those unmanageable systems.

3.20.20 1:38am - (Replying to @jules__shen) Impossible! How can a skill that governs so much of our every day thinking and emotions be "very hard"? Impossible! Indeed, the 3-step process of computing counterfactuals is easy. #Bookofwhy

3.19.20 7:15pm - (Replying to @tdietterich and @zacharylipton) Not familiar with the specifics, but would like to mention that society paid dearly for brilliant perpetual motion machines, proposed, pursued, funded, argued in courts etc. many decades after Helmholtz (1847) paper on conservation of energy. Seek balance & principles! #Bookofwhy

3.18.20 8:21am - (Replying to @100trillionUSD @pierre_rochard and 5 others) No. I said: IF cointegration analysis aimes at distinguishing spurious and non-spurious relationships, then it must be based on causal assumptions and, thus far, I have not seen the literature on cointegration explicate such assumptions, nor use causal vocabulary.#Bookofwhy

3.17.20 7:34pm - (Replying to @pierre_rochard @ercwl and 2 others) But what IS this remarkable relationship? Can you show it in just 3-4 variables and 3 time slots? I would like to add it to my list of "remarkable" relationships of correlated variables. If it is truly remarkable, it should be demonstrable in 3-4 variables. Curious #Bookofwhy

3.17.20 7:26pm - (Replying to @JohnMarkoff2 @ercwl and @100trillionUSD) If cointegration is a statistical property of time series then @btconometrics is right, it does not imply causation. See Whether it "smells" causation depends on the assumptions made by the smeller; they may be plausible, but we need to see them #Bookofwhy

3.17.20 7:14pm - (Replying to @pierre_rochard @dopamine_uptake and 5 others) I am yet to find a signal that a certain "correlation is not spurious" which is not based on causal assumptions. So, has any of its advocates been able to explicate those causal assumptions? I know that Granger was not able, see Who was? #Bookofwhy

3.17.20 4:03pm - (1/ ) Recent exchange with theory-averse readers compels me to retweet a slogan: "I would rather follow Eratosthenes and measure the radius of a legendary turtle than fit curves in a Babylonian observatory." It refers to Greek science in Toulmin's book "Forecast and Understanding"
3.17.20 4:03pm - (2/ ) (see, last page) and his observation that it takes a theory to design a meaningful experiment. Moreover, we are fortunate to live in an era where the "understanding" part of Toulmin's title can be emulated on a computer and tested against its rivals.
3.17.20 4:03pm - (3/ ) I am speaking of course about SCM which, to the best of knowledge, is the first system to deserve the title "understanding", due to its ability to pass the mini-Turing test on all three levels of human understanding (#Bookofwhy Ch. 1): Seeing, Acting and Imagining.
3.17.20 4:03pm - (4/ ) Its opponents do not usually offer computational alternatives, preferring instead to speak in generalities about the superiority of data over "false narratives." Still, by the time they are ready, we should be able to run the first empirical test of the "Scientific Method."

3.17.20 3:16pm - (Replying to @Saravanan_CU) The main disparities that I would include are disparities among gv't policies. Still, transportability should be used whenever we generalize across populations.Unfortunately the theory is only one decade old and analysts in power take 3 decades to shift paradigms. #Bookofwhy

3.17.20 9:40am - (Replying to @sean_a_mcclure and @HarryDCrane) The opaque culture you prefer reminds me of Babylonian astronomy in Toulmin's "Forecast and Understanding" (see, last page) It may get us to Babylon, but not to Athens; I prefer the footsteps of Eratosthenes and measure the turtle's radius #Bookofwhy

3.17.20 12:57am - (Replying to @raymondshpeley) I like it too. And am going to keep it. I truly feel for those who judge #Bookofwhy by how it confirms what they knew already, instead of rejoicing the tools it provides for doing today what they wanted and could not do yesterday. Boy! how much fun they are missing!

3.16.20 9:16pm - On the contrary! I tell students: "formality amplifies common sense." Look at common sense logic before and after Boole. Look at reasoning with uncertainty before and after Bernoulli + LaPlace. #Bookofwhy

3.16.20 8:49pm - Commiserating with BDS cronies - blazing beacons of deceit. They smile while lying and smile when caught. Do they really know the difference?

3.16.20 6:29pm - (Replying to @DKedmey and @HarryDCrane) One important caveat. If you know nothing about your past behavior except that you are like anyone else in a given subpopulation S, then the personal question: "Would I recover if I take this drug" can be answered from population-level data: find P(recover|do(drug),S). #Bookofwhy

3.16.20 6:00pm - (1/ ) Back to students of CI and other sapiens fascinated by logic, counterfactuals, and everyday language. Here is a commonsensical, everyday notion called "NEED", as in "lets first test people in greatest NEED". We take it for granted that, if the need arises, analysts would
3.16.20 6:00pm - (2/ ) be able to define what "being in NEED" is, and how to quantify the degree of NEED. It now downed on me that this simple notion is intrinsically counterfactual, hence only those familiar with PNS would be able to agree on how to quantify it, given data and commonsense knowledge
3.16.20 6:00pm - (3/ ) I have nothing exciting to say to those who get angry at #Bookofwhy. Knowing everything they are not fascinated by anything, and miss so much of the fun of discovery.

3.16.20 5:09pm - (Replying to @MalikMashail and @McDonalds) I tried not to get too personal. Oh, almost forgot my logo: #Bookofwhy, sorry.

3.16.20 4:54pm - (Replying to @DKedmey) Well put! I could not express it more clearly. In addition, this difference translates into (i) individual level vs. (ii) population level inferences. To guess whether "Joe would have reacted differently" I need to undo what I know about Joe. #Bookofwhy [no off @HarryDCrane etal]

3.16.20 4:32pm - (Replying to @HarryDCrane) FYI, and for all those offended by my tweet: please note: The tweet was addressed to "students of CI" who have been estimating PNS for e-commerce and drug approval. Identifying "those in greatest need" is another application we should keep in mind.

3.16.20 4:13pm - I end my tweets with #Bookofwhy because, so I am told, many readers filter their tweets by this tag, and I do not want them to be left out of the discussion. It is not meant to suggest that the book "solves every problem"; an epsilon of commonsense would rule out such suggestions

3.16.20 3:40pm - (Replying to @DavidPoe223) Right on the nail. It is now 30 years since I started asking regression analysts: "What puzzle a regression is supposed to solve?" Some say:"Given information X, we want to predict Y". Fine. But some say "We want to understand, to decompose, to make sense". Dont get it.#Bookofwhy

3.16.20 3:25pm - To students of CI, Coronavirus is a reminder that "people in greatest NEED" is a counterfactual notion; the degree of "need" is the probability of "helped if treated & doomed if not", dubbed PNS in Causality p.286, and tightly bounded in #Bookofwhy

3.16.20 2:57pm - (Replying to @DavidPoe223) In Simpson's paradox the puzzle is simple: We cant conceive of a drug that is good for men, good for women, and bad for a person. What is the puzzle that "multilevel and fix effects" models are trying to resolve? #Bookofwhy

3.16.20 7:02am - (Replying to @DavidPoe223) I am still curious: What is being sought? What is to be estimated? What is it that does not make sense before you start writing down equations? #Bookofwhy

3.16.20 5:59am - More from Michael Levitt on why the spread of Coronavirus is slowing #Bookofwhy

3.16.20 4:03am - (Replying to @JRubinBlogger) And we can safely assume she won't be Linda Sarsour. If I were Biden, I would make this point loud and punchy.

3.16.20 3:54am - (Replying to @rodakker @tudorcodes and @imleslahdin) I hope what I mean by "causal" coincides with yours, causality is a human language. As to Jaynes, he was sold on probabilities, so I do not believe he can define "causal arrows" better than other Bayesians, who are semi-alien to causation: #Bookofwhy

3.16.20 12:10am - (Replying to @rodakker @tudorcodes and @imleslahdin) This happened to be a Causal Network, even though we are only interested in probabilities. For a non-causal network of the same physics, consider AND--->D1---->D2 plus AND--->D2. It contains non-causal arrows which are necessary to get the probabilities right. #Bookofwhy

3.15.20 3:14pm - (Replying to @tudorcodes and @imleslahdin) All data are generated by some causal processes, hence causal diagrams can always be used to capture the statistical features of the data, provided we allow for hidden variables (& selection B). To save the labor of dealing with hiddens, you may use undirected graphs. #Bookofwhy

3.14.20 10:43pm - Of all the statistical analyses of the C-Vir outbreak, the most optimistic evaluation was given by (Nobel) Michael Levitt, in an interview to China Daily: His technical paper, tables, curves and figures can be viewed here #Bookofwhy

3.14.20 6:04pm - (Replying to @svonava @Joshyblogz1 and @lexfridman) This wikipedia article on counterfactual thinking was written before the algorithmization of counterfactuals. Someone ought to tell the authors to reexamine those theories because much of what they describe can now be done by machines, eg., or #Bookofwhy

3.14.20 1:58pm - Why soap? And why it matters? Statistical analysis of the Coronavirus outbreak is illuminating, but the most useful article I've read in the past two weeks is this: And it has a touch of #Bookofwhy in it.

3.14.20 11:37am - Causal diagrams capture a small but important chunk of human understanding. To their credit, they give us a friendly and formal object from which we can derive what Keynes probably meant by "the broad reasons why past experience was what it was". #Bookofwhy

3.14.20 10:48am - (Replying to @mgaldino and @mendel_random) Barring a few exceptions, of course.

3.13.20 12:59pm - (Replying to @ganesh__s and @eliasbareinboim) Sure! Even static physical equations (eg Hooke's Law) convey counterfactual information. But you have to be explicit how to simulate "undoing", e.g., "what if event E did not occur?" #Bookofwhy

3.13.20 11:39pm - (1/ ) On curve fitting vs. science, @eliasbareinboim showed me a beautiful quote from John Locke (1960, Essay VI): "By the color, figure, taste, and smell and other sensible qualities, ...we cannot tell what effects they will produce; nor when we see those effects can we so much as
3.13.20 11:39pm - (2/ ) (Replying to @yudapearl) guess, much less know, their manner of production." Locke understood in 1690 that the key to human understanding, knowing "their manner of production," cannot be inferred from data. #Bookofwhy

3.12.20 1:47pm - The March issue of the Philosophical Transaction of the Royal Society celebrates 50 years to Price Equation, which describes how genes change over time. I was pleased to find two papers on causal interpretations of the equation.

3.12.20 1:28pm - (Replying to @ewerlopes) Yes. I purchased a new kindle version and, aside from their usual sloppiness in handling equations, they have implemented almost all the errata. #Bookofwhy

3.12.20 5:36am - (Replying to @Stebbing_Heuer @regev_nir and @BernieSanders) My general philosophy is that every experience, however subjective, is a piece of information that should be integrated with other sources to forge theories. I don't know anyone among my colleagues who is not disgusted at the way @BernieSanders is selling his soul to angry voters

3.12.20 5:21am - (Replying to @NNrehman) Sure. But please consult the Errata list here:

3.12.20 4:46am - The day we were all been waiting for has arrived!!! Our publisher informs us that the new, revised and corrected printing of Primer is now out at all US outlets. To make sure your copy is "revised", check that the "about the authors" page cites #Bookofwhy

3.12.20 4:39am - (Replying to @Stebbing_Heuer @regev_nir and @BernieSanders) My personal, first-hand acquaintance with my compatriots, the American people, and their deeply ingrained respect for leaders who do not sell their heritage for a few angry votes.

3.12.20 4:31am - (Replying to @jasndoc and @Farzad_MD) A highly motivated reason to examine how modern causal analysis can be harnessed to assist a task which appears purely associational but involves fusing together diverse sources of information from diverse populations. #Bookofwhy

3.12.20 1:34am - (1/ ) (Replying to @regev_nir and @BernieSanders) "Think not with thyself that thou shalt escape in the king's house, more than all the Jews. For if thou keep thy silence at this time, relief will arise to the Jews from another place, but thou and thy father's house will perish". Mordechai would'nt use such harsh words, unless
3.12.20 1:53am - (2/ ) (Replying to @yudapearl @regev_nir and @BernieSanders) he sensed reluctance on Esther's part to plead with the King despite timing considerations ("I wan't called for 30 days"). I think Michigan voters felt the same about Bernie's selling Israel, the culmination of his people's history, for a few angry votes, promised by Rashida.

3.10.20 11:26pm - (Replying to @AnneHerzberg14 @richard_landes and @stan_state) B. Franklin's words keep ringing in my ears: "He who sleeps with dogs will rise up with fleas." And the Mishna says: "Leaders, watch your words! Lest your followers misinterpret them!" Poor Bernie, will he ever understand the dangers of bad company?

3.10.20 10:57pm - (Replying to @AnneHerzberg14 @richard_landes and @stan_state) It is not funding that the faculty of @stan_state should worry about, but their reputation. As much as I am telling myself "dont blame the grass for one bad weed", it does reflect on their mentality, what they tolerate as "scholarship" and what they feed students. STANISLAUS

3.10.20 10:32pm - (Replying to @regev_nir and @BernieSanders) Esther did need this. She was about to brush it off with: "The timing is not right, etc etc." Mordechai reminded her: "Perhaps history elevated you to this position for a purpose". Bernie had no one to remind him. I tried, but I doubt he can hear the bells of history. Poor Bernie

3.9.20 1:21am - Poor Bernie, he thought by abandoning Israel he would win the hearts of his Zionophobic followers. Alas, they now reject him as "rich Jew": He forgot what Mordechai told Queen Esther: "You wont save your skin by abandoning your people". @BernieSanders

3.9.20 10:11pm - (1/ ) Columbia President Lee Bollinger has issued a statement condemning antisemitism on campus. Kudos! But he evidently has not read my advice To stop BDS hostilities it ain't enough to decry anti-semitism; you've got to expose what
3.9.20 10:11pm - (2/ ) BDS stands for: the elimination of Israel, and the expulsion of Israel-inspired students (and faculty) from campus life. And you've got to express how you, personally, feel about what they stand for, as President Martha Pollack of Cornell did: @Columbia

3.9.20 2:03pm - (Replying to @charleendadams) Never abandon babies, with or w/0 waters. It's good for science to bemoan what we need: triangulation, external validity, selection bias, missing data, etc etc. Bemoaning loud enough and long enough may get someone attention, lest it becomes a habit that avert us from doing it.

3.9.20 5:28am - (Replying to @kurtosis0 and @BernieSanders) Repeating: "Abandoning Israel" means supporting, endorsing, and giving platform to those who seek Israel's destruction. You can abandon Israel being an American or a Hottentot, all you need is moral frailty and Zionophobic voter base

3.9.20 5:10am - (Replying to @jmourabarbosa and @BernieSanders) Abandoning Israel means supporting, endorsing and giving platform to those who seek Israel's destruction. Simple! Simpler even than causality.

3.9.20 3:34am - The word "triangulation" has been used by many to express dissatisfaction with established methodologies and romantic aspirations for something new. This paper however, recognizes formal ways of triangulating information: #Bookofwhy

3.9.20 3:06am - (Replying to @Stebbing_Heuer @kurtosis0 and @BernieSanders) You got it perfect. Being Jewish is not genetic, nor religious, nor being a victim of the holocaust. It is identifying yourself with the history and destiny of a collective calling itself Jewish. Anyone can acquire it. E.g., Large segment of former Soviet citizens now in Israel.

3.9.20 1:32am - (Replying to @kurtosis0 and @BernieSanders) And Queen Esther was Persian. Truman and Kennedy were American too, but would not admire Zionophobic racists like Linda Sarsour and Ilhan Omar.

3.8.20 6:27pm - (Replying to @GeoffWinestock) I did not realize you have a question. I agree that the British could have done many things (eg. allow movement on the Danube to the Black Sea) but this does not absolve the Mufti from the crime of strangling European Jewry by the anti-British anti-Jewish riots of 1936-39,

3.8.20 4:59pm - Today is international women's day, coinciding with the holiday of Purim, the day that Queen Esther said: I must do it, whatever comes. I salute the Israeli female soldiers who served with me in the army (1953) and kept my hopes high until this very day:

3.8.20 4:33pm - It is the first time that I hear this music and I see the charming smile of Al Husseini, the man who prevented my grandfather from escaping Hitler, the father of BDS (1936 boycott of Jewish products) and the creator of Zionophobia. May history remember him for his deeds.

3.7.20 2:01pm - (Replying to @RokoMijicUK and @BonbonFork) No qualms about "big jumps", but when I characterize a tool, I say what the tool delivers, not how it operates. #Bookofwhy

3.7.20 9:03am - (Replying to @ChrisAdamsEcon) McFadden's random utility model is one model in which we can compute certain POs. RCT is another. Speaking in general, however, given a structural model M, which PO's can be computed, and how? #Bookofwhy

3.7.20 8:04am - (Replying to @ChrisAdamsEcon) That is why I prefer "first person" comments, rather than hearsay. Can you state: "To me, Phil's views resolved several questions that I had: (1), (2)...." Or: "If you havn't read Phil's slides you might be tempted to believe (1),(2).. Not so!" #Bookofwhy

3.7.20 7:44am - (Replying to @FriendlySceptic) Speaking of intellectual honesty, the word "equating" is out of place. I said: When you endorse Zionophobic racists like Linda Sarsour and Ilhan Omar, you should expect other racists to smell blood and yell: "Free season!" Havn't heard Sanders' defenders defend these endorsements

3.6.20 9:41am - (Replying to @TheMongoose4) Never mind what right-wing Jews are saying. Listen to what left-wing Jews are saying: When you endorse Zionophobic racists like Linda Sarsour and Ilhan Omar, you should expect other racists to smell blood and yell: " free season!". Signed: a disappointed Bernie sympathizer.

3.6.20 9:12am - (Replying to @roieki) Chas V'Chalila!! Never "BaMachane", and never "Nazism". But when you endorse Zionophobic racists like Linda Sarsour and Ilhan Omar, you should expect other racists to smell blood and yell: " free season!"

3.6.20 8:38am - Schoking? Yes, Surprised? No. I believe it was Benjamin Franklin who said: "He that lieth down with dogs shall rise up with fleas". When you open your house to haters you can't screen them for table manners.

3.6.20 6:18am - (Replying to @ChrisAdamsEcon) We discussed Haile's slides here, on Twitter, and I did not get the impression that economists agree with him. Correct me if I am wrong. Also, help us find where economists define PO's from structural equations. The closer I got was Heckman: #Bookofwhy

3.5.20 11:51am - Econometrics has two camps: Structural and quasi-experimental. The former say: PO comes from structure, but don't tell us how. The latter resist structure, and pretend PO resides already in one's mind. Both camps needs a shake, I tried here, & #Bookofwhy

3.5.20 11:44pm - (Replying to @JDHaltigan) Thanks for educating me on this new measure. I'm still trying to figure out when I would feel that strange need to hold Z constant for just X. BTW, the expression "hold Z constant" is misleading, it should be "selecting only cases for which Z=z". #Bookofwhy

3.5.20 11:27am - (Replying to @jiaming_mao and @causalinf) Symmetric? No. Each can be separately derived from a completely specified SCM (a set of structural equations). It is so nicely described in Primer, and it is available for free:, Why mystify? #Bookofwhy

3.4.20 11:34am - (Replying to @MrTeshnizi) Good question. I think we are born with a template for gathering and using causal knowledge, which DL agents do not. If they do, they should be able to pass the mini-Turing test of #Bookofwhy chapter 1.

3.4.20 4:49am - I'm absolutely delighted to be elected Honorary Fellow of the Royal Statistical Society, an institute that has been rather lukewarm towards causality in the past. I see it as an invitation to all of us, causal enthusiasts, to meet the challenge of becoming mainstream. #Bookofwhy

3.4.20 2:36am - Better in the sense of having lower asymptotic variance.

3.4.20 12:30am - Explanations of what? Examples of what? Sorry, I am a foreigner, please speak slowly to me.

3.4.20 12:04am - A reader just resurrected this yr-old tweet, and reminded me that my question is still unanswered: Which estimator is better? The ML estimator of the product or the product of the ML estimators?

3.3.20 11:53pm - My heart goes to MN voters. Who will protect their children from the Zionophobic hatred spewed by @BettyMcCollum04 ? Who will protect their reputation from the thought they might share her mentality? My heart goes to Minnesota voters.

3.3.20 9:29am - (Replying to @mariabloec @jonasobleser and 5 others) I have a conciliatory ending for this debate: The notion of "Well-definedness" is ten times more ambiguous than the causes it tries to disambiguate. #Bookofwhy

3.3.20 8:46am - (Replying to @mariabloec @jonasobleser and 5 others) I cannot think of a well-defined interventions for earthquakes and, still, their causal effects on property damage is not vague. I discuss it here and here "Well definedness" suppresses causal thoughts. #Bookofwhy

3.3.20 8:36am - I love the ending of this old post: "Once tried - always used". Thanks for re-posting. It applies to so many tools of causal inference. #Bookofwhy

3.3.20 6:22am - An arrow between A-B would be a different story, which we could try later as the hard part of the test. #Bookofwhy

3.3.20 12:22am - It is an honor to be in the company of these stellar authors, Thanks

3.2.20 11:23pm - My advice to readers who are asking me to help them recruit students and faculty in ML: State explicitly that you seek candidates in the CI area. This would signal to candidates that your department is serious about changing the causality-free tradition of ML education.#Bookofwhy

3.2.20 12:37pm - (Replying to @BlueManifold and @dmonett) Indeed, now that I re-read it, they do emphasize the importance of causal and counterfactual reasoning. I did not pay attention to it in 1969, being deep into heuristic search. It is only when I saw causality as the force behind Bayes networks that I asked: what is it? #Bookofwhy

3.2.20 7:46am - (Replying to @mktopuz) It's easy to check using the mini-Turing test on the firing squad (#Bookofwhy chapter 1). "Would the prisoner still be dead had soldier-1 refrained from shooting?"

3.2.20 7:23am - The next step is to ask: what kept the 1912 guy from recognizing the importance and/or generality of his/her observation, and turn it into a "theory of 1912"? Language has lots to do with it. #Bookofwhy

3.2.20 7:14am - (Replying to @BlueManifold and @dmonett) Computer science is the only discipline I know, in which people understand that languages are here to serve a purpose, to be extended and replaced, not to be religiously wedded to, till death do us part. #Bookofwhy

3.1.20 11:41pm - For curious readers, the intrasitivity of correlations, treated as an "issue" in @nntaleb paper "fooled by correlation," is an invariant feature of colliders. In X-->Z<--Y we have Cov(XZ)>0, Cov(ZY)>0 yet Cov(XY)=0 regardless of what numbers we spray on the arrows. However,

3.1.20 11:41pm - DAGs enjoy a weaker form of transitivity: If X and Y are each dependent on Z, then they must be dependent on each other, either marginally or conditional on Z. See "Probabilistic Reasoning.." (1988, p.129). It is not an immutable property of probabilistic dependence #Bookofwhy

3.1.20 3:29pm - "English" is great to describe a phenomenon and show how it is reflected in data. We need a mathematical language to fully understand a phenomena, so as to meet the criteria proposed here vis a vis Simpson's paradox. (English description=1899) #Bookofwhy

3.1.20 3:13pm - I like the way you put it: "formalize the intuition that you take for granted". But I dont foresee "a big blunder" for those who resign to taking certain intuitions for granted. The only problem might be repeating analyses on intuitions that come from the same source. #Bookofwhy

3.1.20 2:36pm - This is precisely the problem of answering counterfactual queries from group's Data, see for a gentle introduction and a tool kit. The "threats" paper you cite should help us motivate modern "remedies", thanks. #Bookofwhy

3.1.20 2:12pm - (Replying to @nntaleb) Your paper shows how certain expectations (eg additivity, transitivity) are often violated. I ask: whence those expectations come from? The logic of causal intuition provides insights into this question. BTW, MI is also intransitive; is it bad? good? surprising? Why? #Bookofwhy.

3.1.20 8:07am - (Replying to @AngeloDalli @AdanZBecerra1 and 2 others) It depends on the granularity of our variables. If I measure the movement of my coin-flipping finger in mm, the outcome is causally independent of that movement, but if I measure it in Angstroms, it's a different story. #Bookofwhy

3.1.20 6:16am - It is fascinating to watch how traditional probabilists handle puzzles that emerge from causal reasoning (in this case the collider structure) lacking a language to capture causal relations. How do we tell them about the causal language w/o offending them? @nntaleb #Bookofwhy

2.29.20 2:03pm - Indeed our conception of cause and effect has only ONE point of contact with do-calculus - the starting point. Do-calculus insists on starting with human conception of reality, faithful to the way it is stored in the mind. The rest is mechanical, like algebra. #Bookofwhy

2.29.20 1:30pm - (Replying to @tmorris_mrc and @y2silence) We can't allow any reader to be left with "low points". The quote is about human conception of cause effect relationships. While external tools for answering certain causal questions may be absolutely necessary, this does not make them part of the conception. In the case of RCT,
2.29.20 1:41pm - we can read text after text of scientific explanations of biological phenomena - what leads to what, what prevents what etc etc - with not a single mention of RCT. See eg the Appendix to RCT is a tool for interrogating, not explaining nature. #Bookofwhy

2.29.20 1:16pm - Why does it matter if Zionophobia IS antisemitism or just another form of racism. The question is how genocidal it is in its aim and how deceptive it is in its advocacy.

2.29.20 5:41am - Our conception of cause and effect is almost always non-deterministic (eg,. careless driving causes accidents), so what? It still can be reasoned about ,formally and informally, without RCT's. #Bookofwhy

2.29.20 5:02am - Got a neat brain teaser from a reader of #Bookofwhy regarding Game 2, page 159: Controlling for D would constrain the mediator E, so why is it legitimate? It's good for a homework, for it ain't trivial. The answer is in the Appendix of Enjoy.

2.29.20 4:18am - Thanks for the compliment. All we need to do now is to convince the establishment.

2.29.20 4:09am - Agree. The computational approaches rangesfrom "simple statistical models like linear regression, to... [more statistical models] like convolutional neural networks". Hoping the 2nd edition snaps out of Rung-1 to methods that interpret data. @DShaywitz #Bookofwhy

2.29.20 3:42am - Zionophobe is one who does not believe in the right of the Jewish people to a homeland, in the context of "2 states for 2 peoples, equally legitimate and equally indigenous". Examples are: Palestinian leadership, educators, clerics, BDS activists and their European supporters.

2.29.20 3:32am - I spread messages of peace and equity, summarized as: "2 states for 2 peoples, equally legitimate and EQUALLY INDIGENOUS." I am risking my academic status to spread this message, because others don't, and many of my like-minded colleagues are prevented from openly doing so.

2.28.20 7:18pm - (Replying to @jos_b_mahoney) No. Apology. I do not know this work.

2.28.20 6:28am - I have always thought that "object-property" relationships (ge, birds fly) require new semantics, not captured by causation. This paper proves me wrong: Time to preserve old beliefs and time to revise them (said King Solomn). #Bookofwhy

2.28.20 5:42am - I would not call a statistical equality "constraint". It is more a "happening", or "coincidence" in the observed data than a constraint imposed by the model's assumptions. #Bookofwhy.

2.28.20 5:37am - Of course! If the outcome of a random coin makes me win a dollar, then it is certainly causal. You are probably asking whether the dependence of the coin outcome on the way I flip it is causal. Classical mechanics says: Yes, despite the absence of prediction/control #Bookofwhy

2.28.20 3:00am - Sure! I would lump all such factors under "metabolism" and add it as a common cause of W_I and W_F, on top of the arrow W_I ---> W_F. #Bookofwhy

2.28.20 12:37am - A Zionophobe is he who says that Israelis, as biological objects "have the right to live in peace" , but will never say that Israel, as a sovereign state of the Jewish people, has the right to live in peace.

2.28.20 12:25am - (1/ ) On Dec. 9 1666, the Rabbis of Constantinople excommunicated Shabbetai Zvi, a self-proclaimed messiah who led a sect of X-Jews to speak against the greater Jewish community. In Feb. 24 2020, 347 Rabbis rebuked Bernie Sanders for speaking against the greater Jewish community .
2.28.20 12:25am - (2/2) If I were a Rabbi, I would move to pronounce Bernie Sanders a X-Jew. This would not diminish his standing as a Messiah, but it would revoke his credentials to speak as a Jew.

2.28.20 12:08am - Three days ago, 347 Jewish Rabbis rebuked Bernie Sanders for maligning AIPAC and Israel. I have not heard any of them say: "Hey, Bernie you either are, or act like a Zionophobe, and this does not reflect well on your moral character!"

2.27.20 2:49pm - In Lord's (1967) paradox, one constraint in his own plot is, cov(S, WI)=cov(S, WF), where WI & WF are two repeated weights. To embrace it in his DAG, @yudapearl had to claim that i) S->WI (a) equals the sum of ii) S->WI->WF (ac) & iii) S->WF (b): a=ac+b. Not super satisfying 2 me

2.27.20 9:13am - I do not usually engage in futuristic speculations, but this interview with Oren Etzioni on "How to know if artificial intelligence is about to destroy civilization" strikes me as a serious thought provoking session. #Bookofwhy

2.26.20 5:00pm - Sure. The best place is this personal story from the Jewish Journal I also have some photos that go with it: Note the neighborhood bully, it's me, no messing around.

2.26.20 9:32am - Who is a proud Jew? S/he who calls Bernie Sanders a ZIONOPHOBE, in public. As far as I know, no one has done it yet! May I have the honor? Proudly so!

2.26.20 8:19am - For readers who are asking about software tools to implement do-calculus, data-fusion and other goodies mentioned in @Bookofwhy, here is

2.26.20 7:42am - No Jew is a proud Jew who fails to call @SenSander a Zionophobe, in public. And, as far as I know, no one did. May I have the honor? Proudly so!

2.26.20 7:35am - Sharing a rare and breathtaking view of the Dead Sea coastline, which I, as a boy, knew only as a sub-burned yellow desert.

2.26.20 7:09am - (Replying to @yudapearl @AngeloDalli and 3 others) Halpern's complete axiomatization of counterfactuals, and Spirtes proof that d-separation holds in linear feedback systems. But we lack an algorithmic method, like do-calculus to tell us, in the non-parametric case, when a query is identifiable and when it is not. #Bookofwhy

2.26.20 7:04am - (Replying to @AngeloDalli @ylecun and 2 others) Depends on what we mean by "solution". We have a satisfactory semantics, given by the 3-step process, once the model is fully specified. This is the logical basis for next step: identification. It isn't as easy as in recursive models, but some nice results do exist, among them

2.26.20 12:59am - Readers of #Bookofwhy who are in the healthcare practice or research will find this comprehensive survey of Bayesian Networks to be most informative: It covers predictive, diagnostic as well interventional applications.

2.25.20 9:27pm - Don't under-rate yourself. If you read #Bookofwhy you already know more than the millions who have not read it, and that's quite a lot.

2.25.20 9:22pm - (Replying to @AngeloDalli @ylecun and 2 others) There is a paper by Richardson and Lauritzen which gives cyclic interpretation to chain graphs. #Bookofwhy

2.25.20 9:18pm - I had to write them in this order because, lacking courage and audacity, I could not face the public till I was sure I was right - a personality weakness. But now that I am sure, I do not buy the wimpy "multiple approaches" myth - another personality weakness. #Bookofwhy

2.25.20 8:59pm - (Replying to @yudapearl @mribeirodantas and @diomavro) on the "linear microscope", and, which clearly distinguishes the two tasks and shows that the partial regression BETA(yx|z) is independent of the value z of the variable Z. This is no longer true in non-Gaussian models #Bookofwhy

2.25.20 8:49pm - Bias in predictive tasks occurs in finite-sample estimation, but not asymptotically, as in causal tasks. Unfortunately, most texts on regression analysis confuse the two, as do texts in Economics and Social Science. I must therefore overcome modesty and refer you to my papers

2.25.20 8:31pm - (Replying to @mribeirodantas and @diomavro) Bias due to bad-control only hurts us in causal tasks, where our target is a causal effect. In purely predictive tasks there is no such notion as "bias", and traditional wisdom says: the more information you have the better the prediction. True in linear Gaussian model #Bookofwhy

2.25.20 12:39pm - In my blog comment, in case you could not find it, I provided three relevant links: 1. 2. "Bayesianism and Causality, or, Why I am Only a Half-Bayesian" -- 3. Causality, section 11.1.1 #Bookofwhy

2.25.20 10:36am - O.A. Arah has called my attention to a lengthy discussion on Gelman's blog: concerning a paper on "causal inference with Bayes Rule" I have added a comment to the former, and will examine the latter. #Bookofwhy

2.25.20 8:35am - The channel you are now using, by following @yudapearl , is an educational channel, dedicated to the scientification, algorithmization and democratization of causal inference. #Bookofwhy

2.25.20 3:23am - (Replying to @yudapearl) On this occasion, let me also recount the availability of two resources. (1) A searchable file of all our 4,263 Tweets., and (2) A listing of all UCLA-CSL papers and technical reports, dating back to 1965. Happy sailing #Bookofwhy

2.25.20 3:23am - My faithful Twitter counter says that, today, our educational channel has garnered the attention of 30K followers. This encourages me to continue our journey towards the scientification, algorithmization and democratization of causal inference. Thanks for the encouragement.

2.25.20 3:07am - (Replying to @AngeloDalli @ylecun and 2 others) Do you mean cyclic causal paths? (ie. feedback loops)? If so, please examine causality p. 215-217. There is a robust solution, though the identification phase is trickier. #Bookofwhy

2.24.20 5:48pm - (Replying to @jim_savage_ @steventberry and 5 others) For structural economists, an agreement on "non-parametric" should come easy, it is a fancy surrogate to "structural", namely, properties that emerge from the bare structure of the model (excluding for example shape restrictions, or distributional assumptions) #Bookofwhy

2.24.20 5:20pm - (Replying to @steventberry @jim_savage_ and 5 others) The "honor" of structural models is currently being restored through (1) clear definitions (which precede identification) and (2) tools (eg graphical) for liberating models from their non-structural burdens and unleashing truly non-parametric identification properties #Bookofwhy

2.24.20 4:52pm - (Replying to @steventberry @jim_savage_ and 5 others) As a student of philosophy I see it as a challenge to track down sources of confusions in various disciplines and, although econ. is still an enigma to me, I can see interesting patterns in econometric thought, going from inadequate notation to inadequate definitions. #Bookofwhy

2.24.20 1:26pm - (Replying to @steventberry @jim_savage_ and 5 others) It feels great to be in agreement: "The answer is: structural." Perhaps you also share with me another conjecture, that much of the confusion in Econ emanates from not having a crisp, undisputed definition of Y_x, in terms of a (fully parametrized) structural model. #Bookofwhy

2.24.20 12:17pm - (Replying to @jim_savage_ @steventberry and 5 others) Thanks for re-posting. The word "counterfactual" appears dozens of times in these slides. Perhaps they can clarify how traditional econ defined Y_x(u) or, even more ambitiously, which of "structural vs. reduced" form is more convenient for evaluating E[Y_x|X=x', Y=y']? #Bookofwhy

2.24.20 12:16am - I am still not clear if there is anything new in p. 97 that econ students can use, or is it all 3rd-week freshman class. I was ready to offer a big apology for wrongly believing in the former, but I am not getting a definitive affirmation of the freshman model. Help! #Bookofwhy

2.23.20 5:58pm - (Replying to @steventberry) I am counting the steps needed AFTER identification. How do we compute the expected salary of Joe who now has skill level Z=1 had he had one more year of college (see Fig. 4.3 in I do not know how to skip the 3-steps of p.97, please teach me.#Bookofwhy

2.23.20 3:21pm - (Replying to @steventberry and @PHuenermund) Sorry this tweet got out of thread

2.23.20 3:18pm - Yes, I recall a lively discussion with Heckman on the definition of "effects": But counterfactuals like E(Y_x|x',y')are a bit more complicated, needing a 3-steps computation, which I have not seen in the econ. lit. Perhaps you do it in one step? #Bookofwhy

2.23.20 1:25pm - I am happy to hear that counterfactual questions are now solved in freshmen econ. classes, and it was only my bad luck that I could not find anyone who could solve it in 1998. I am still curious, though, what definition of the counterfactual Y_x is used ? #Bookofwhy

2.23.20 5:51am - I hate to disagree, but I must note my disagreement with the analogy. Econometrics starts with a (structural) model that every economist can comprehend. PO starts with ignorability assumptions that almost NO epidemiologist can comprehend. See 3 bullets

2.23.20 12:16am - I am still not clear if there is anything new in p. 97 that econ students can us e, or is it all 3rd-week freshman class. I was ready to offer a big apology for wrongly believing in the former, but I am not getting a definitive affirmation of the freshman model. Help! #Bookofwhy

2.22.20 6:19pm - We are in receipt of a course on "Superforcasting",, part IV of which says: "Superforcasting requires countefactualizing". This means that an algorithm equipped with a causal model, should out-predict one based on data alone. True, but how? #Bookofwhy

2.22.20 1:32pm - Commenting on my "Two new weapons.." a reader claimed: "Protestantism led the way in Zionism, not Jews". Reply: Not so; my grandpas prayed 3 times a day "Restore us in sovereignty to our land" (Hamazon)

2.22.20 11:41am - (Replying to @pierre_rochard @dopamine_uptake and 5 others) I am eager to agree, but I do not know what "s2f shills" is. Culturally deprived.

2.22.20 11:27am - (Replying to @dopamine_uptake @TheStalwart and 5 others) If by "useful" you mean "causative", then the question need be rephrased to read: What causal assumptions are needed to make Granger causality "causative"? I recall (vaguely)

2.22.20 11:18am - (Replying to @pierre_rochard @dopamine_uptake and 5 others) Another mathematical result says that there is no way to distinguish spurious from non spurious correlations without causal assumptions. See These ought to have been part of econ. education, but see what happened: #Bookofwhy

2.22.20 9:41am - (Replying to @TheStalwart @dopamine_uptake and 5 others) It is not exactly a mathematical dispute as it is a mathematical fact that one cannot define causal relations in the language of probabilities, nor can one claim causal relations without causal assumptions. The latter are missing from Granger's causality. #Bookofwhy

2.22.20 7:30am - The shameful complicity of the journalistic fraternity will always embitter my memories. Like those "journalists" who cooked up the story of Mohamed Al Dura, whose picture was used by Danny's murderers.

2.22.20 7:04am - Well put. "facelift to Sharansky's 3 Ds"; so badly needed. The 3 D's served their purpose, but they also reinforced the outdated fixation that anti-Zionism is evil only in as much as it contributes to antisemitism, which has spawned and legitimized Zionophobes like Linda Sarsour.

2.21.20 9:07am - "How much the system IMPROVES the likelihood of a good outcome" is precisely the counterfactual term that PNS captures. #Bookofwhy

2.21.20 5:21am - And let's not forget the Passover Hagada: Whoever did not say the following three things in the past week, has not fulfilled his obligations to students on campus: Zionophobe, Zionophobic and Zionophobia. Words count, and silence corrupts.

2.21.20 5:00am - Cornell's President, Martha Pollack, should serve as a role model to all academic leaders, especially to Berkeley's Chancellor, Carol Christ, who said last week: "Jewish students have the right to feel dismay and concern" instead of: "I, personally, feel dismayed and concerned."

2.21.20 3:28am - (1/ ) Legality aside, I would assess how likely I am to encounter a complication never seen by the AI surgeon, which a human surgeon can handle through her knowledge of anatomy. It actually happened to me, when a famous surgeon quoted us 90% success record but, when asked
2.21.20 3:28am - (Replying to @yudapearl) (2/ ) how many cases like THAT have you seen? He said: "None. I have never seen a lung as bad a lung as this." We cancelled the surgery, and my wife is still alive. Remarkably, counterfactual logic permits us to bound the probability that a given patient will benefit from a given
2.21.20 3:28am - (3/ ) treatment, once we have a combination of both experimental and observational data. See tight bounds on PNS (prob. of necessary and sufficient causation)

2.20.20 3:01pm - (Replying to @omaclaren @AdanZBecerra1 and 2 others) Who would care about identifiability if is were not a necessary condition for estimability? And who would care about "a particular class of models" if it were not a faithful representation of your very own understanding of the world? "Estimability" reminds us or that. #Bookofwhy

2.20.20 12:56pm - I define an expression to be "estimable" if it is made up of sums and products of conditional probabilities, and the underlying distributions are non-pathological. #Bookofwhy

2.20.20 12:44pm - It was an honor to speak to Alums for Campus Fairness (not 'algorithmic fairness') and to share with them my two defensive weapons. I hope these weapons will be used effectively by all colleagues and students who are subjected to Zionophobic harassments on their campuses.

2.20.20 10:36am - I can forgive them all, including the last one, is he only did not sell his Jewish soul to Linda Sarsour and other Zionophobes.

2.14.20 6:40am - Today we will say Kaddish in memory of our son, Daniel If you happen to be in Encino, CA, join us at 5pm at Valley Beth Shalom. Else, please join us spiritually. After the Kaddish, please sing with us: Tsadik Katamar Yifrach (Like an Immortal Palm Tree)

2.20.20 6:25am - The gives indeed a good overview of causal inference, albeit from a fairness perspective, especially the CI chapter I'm glad they took Berkeley's sex-discrimination debate as a test case, and built new ideas around it. #Bookofwhy

2.20.20 3:15am - (Replying to @ChengSoonOng @mrityunjay_99 and @suzatweet) Beg to differ. The test is fruitful, necessary, available and has led to only few debates, mostly among traditional statisticians, who could not reconcile to the fact that the most powerful language they knew, prob. theory, cannot do what they always wanted to do (CI). #Bookofwhy

2.20.20 3:08am - (Replying to @ChengSoonOng @mrityunjay_99 and @suzatweet) The test is as formal and crisp as mathematics Herself. See nfGPQ#page=5. Any inference defined in terms of probability distributions of observable variables is Statistical, not causal. #Bookofwhy

2.20.20 2:51am - Bergstein's article raises all the important issues. I would only wish it was titled "what AI can and will do" instead of "what AI cannot do", which reminds me of the old speculations of Penrose, Searle and Dreyfus. #Bookofwhy

2.20.20 2:25am - The key point is "have the architecture" which is an innate "template" to store and interrogate what we learn through experience. What we cannot do having this template we surely cannot do having to acquire it from scratch. #Bookofwhy

2.20.20 2:14am - Glad you asked me, the least biased arbiter. The @Bookofwhy will equip you with elementary tools to solve elementary problems that even the most sophisticated t heories in statistics, logic and NN cannot solve. It's "low-level" on the one hand and super "high-level" on the other.

2.20.20 1:58am - (Replying to @ChengSoonOng @mrityunjay_99 and @suzatweet) What I meant is a chapter on the mathematical framework (called CSM) that unifies all these seemingly different "approaches" or "styles". (BTW Granger causality is not causality). #Bookofwhy

2.19.20 4:47pm - Statisticians are not to be undervalued. Rather, they need to be awakened to the age of causation and internalize the fact that the language of statistics need be extended to adapt to the new age. #Bookofwhy

2.19.20 7:07am - With the number of enemies I have on Twitter, I am not sure my retweeting would serve you well. Watch your steps carefully and ...(almost forgotten).. take a look at the Sequential Backdoor for more pleasant "time varying" journeys in the future.#Bookofwhy

2.19.20 5:47am - If do-calculus is a special case of X, I would be thrilled to see how X derives the front-door formula. It reminds me how, not too long ago, epi folks considere d Rung 1 and 2 and 3 as the same rung, see #Bookofwhy page 152. Should epi students wait another 25 years?

2.19.20 4:38am - Replying with my retweet to keep the thread unbroken, and to add an iota: I am willing to help an interested author put together a comprehensive textbook on MLL that includes a CI chapter. Will do so anonymously and delightfully. #Bookofwhy

2.19.20 4:18am - Where is MML heading? It seems to be heading towards another period of Rung-1 existence, awaiting textbook authors to discover that a unifying mathematics of causality does exist, within which all seemingly different "approaches" have converged. A golden opportunity! #Bookofwhy 6

2.18.20 12:24pm - When I see political scientists learning from epidemiologists, I see signs of progress, especially when they're using graphical models, like here: What I do not understand is why they aren't using do-calculus, to get identification with no tears #Bookofwhy

2.18.20 10:52am - Israel celebrates the disappearance of an island in the Sea of Galilee: On a personal note, this island was completely covered (a latent variable) during my army-service days, 1953-1955.

2.18.20 2:30am - My point is that Jews were not actively "criticizing" revealed truths. "Debates" were forced upon them. What provoked hostility was not the persistence of their beliefs as much as the persistence of their biological existence; Wrong beliefs had to show up as abnormal existence.

2.18.20 2:06am - This new paper, "Conditional Path analysis in singly-connected path diagrams" provides interesting factorization results to the "Linear Microscope" of #Bookofwhy

2.18.20 12:21am - There is some truth to it, as told in #Bookofwhy p.244, but not exactly as Clark describes it. What I learned from Spirtes (aug. 1991) was arrow-cutting for intervention, but the causal interpretation of DAGs came al ready in 1987, eg and others, 88-90.

2.17.20 1:35pm - The protest was legally tolerable yet morally despicable. Berkeley is waiting no w for the Chancellor's personal condemnation of such Zionophobic disruptions on moral, not legal grounds.`

2.17.20 12:18pm - Debra is right in NOT equating the two. By saying: "anti-Zionism IS ANTISEMITISM " one implies that the latter is more despicable than the former and, indirectly, bestows some leniency onto an eliminationist ideology, which is more dangerous than its emotional rival.

2.17.20 7:25am - (Replying to @ghost_of_roger @deborahlipstadt and @IzaTabaro) The only way to cure this misconception is to use the term Zionophobia, which makes people jump to their feet and ask: What do you mean? Tell me more! The gullible don't understand that being anti-Zionist means being ugly and immoral.

2.17.20 7:12am - anti-Zionism is crisply defined, unlike anti-Semitism which invites infinite discussions and disputations. Perhaps because the latter fell into ill-repute after the holocaust, for a short period of time.

2.17.20 7:06am - Really? I am truly delighted. It is such a fitting metaphor, that I thought would help statisticians understand why causal notions exist only in the 3rd dimension, and what they need to do to capture it. Still waiting. #Bookofwhy

2.17.20 6:59am - You sound like a reasonable person, but you speak with Zionophobic slogans like "apartheid policies". I refrain from using such terms for fear of being identified as gullible and misinformed, who buys into BDS propaganda.

2.17.20 4:35am - Logic does dictate that, if you read my elaboration on why the logic of Zionopho bes makes most Jews suspects of criminal activity, unless proven "innocent," lik e those fringe X-Jews (the new Maranos), who think distancing themselves from Is rael would protect them from harassment.

2.17.20 4:22am - Good point. We should really make clear it's not "probabilistic models" but "mod els of reality", as opposed to models of data. I hate to say "causal models", b/ c ML folks would say "It must be awfully complicated". "Reality" on the other ha nd is supposed to be simple. #Bookofwhy

2.17.20 4:03am - Emil Coman just sent me a new toy, a Stata module to do-the-DAGs: Some of our readers might find it useful to have a '1 stop shop' to do things th ey couldn't do before. #Bookofwhy

2.17.20 1:04am - I heard people say "he was a spy", or "he had no business investigating Pakistan " This is the first time I hear someone saying it was "propaganda". The capacity of certain cultures to deny reality exceeds our capacity to be surprised.

2.17.20 12:31am - The reason I said "insurmountable obstacle" is that it cannot be solved by model -free ML methods. The examples in show that the same shi ft in probability may require two different repairs, depending on which structur e caused the shift.#Bookofwhy

2.16.20 9:17pm - Mike, you made my day! Finally, a Jewish leader with a spine! Who recognizes tha t we must change the conversation from whether anti-Zionism is or is not antisem itism to whether it is genocidal or just disgustingly ugly.

2.16.20 8:03pm - (Replying to @MisterSquirrel2 @LomaahhMore and 4 others) What's the point? That distancing ourselves from Israel should protect us? The y (Zionophobes) disrespect Jews who trade their values for social acceptance, so mething they wouldn't do. They will never stop hating us just because "hate" is socially unacceptable.

2.16.20 7:52pm - Dear @BDSreport . I just tweeted this Q. Why don't you use the word "Zionophobia"? and turn it into the ugliest word o n campus? When we use the word "antisemitism" we bestow some legitimacy onto ant i-Zionism, legitimacy that it does not deserve.

2.16.20 7:29pm - In my entire life I havn't heard anyone calling for the elimination of Iran or C hina from the surface of the earth, or calling whoever opposes the elimination " criminal" or subhuman. Linda Sarsour just said: Any one who humanizes Israelis i s on the side of the "oppressor".

2.16.20 7:13pm - This harassment will continue as long as we play their game that, unlike antisem itism, anti-Zionism has forgivable qualities - it doesn't. The harassment will s top when we raise counter-signs saying: "Its worse!" or "Zionophobia kills," per haps even in lectures on Islamophobia.

2.16.20 11:12am - (Replying to @Prof_Livengood @deborahlipstadt and @IzaTabaro) I am not inventing those terms, I am just quoting my Zionophobic colleagues: " Yes, every Israeli who does not support the dismantling of Israel and turning it into a Palestinian-majority state is complicit with the crime of the century." I read what they write.

2.16.20 10:57am - Both intention and material support are a statistical fact. Recall however that the crime is in Israel's very existence, so are the intention and the support -- both are aimed to help Israel survive.

2.16.20 10:09am - I would say "structure" - everything then follows.

2.16.20 9:23am - I don't think we can afford to wait for them to realize that they are committing suicide, for they will drag all of us into the abyss. It's time to close ranks, and save American Jewry from itself -- Israel and Zionism have the inspirationa l power to do it.

2.15.20 10:38pm - (1/ ) The book you mentioned oX.rEbXEZHHE is indeed a penetrating and unparalleled window to the mystery of Jewish identit y. It was created at an emotional moment of history, when one was able to convin ce 150 Jews, from all walks of life, to take time off their busy
2.15.20 10:41pm - (2/ ) (Replying to @yudapearl @kisjdmls and @nprscottsimon) schedules and address a question they would normally push aside: "When you say " I am Jewish" what do you really mean?" The result was an authentic panoramic vie w of how Jews see themselves in the 21st century. We were gratified to know that , 17 years after publication, a book
2.15.20 10:42pm - (3/3) inspired by Danny's last words is still informing and empowering curious minds; quite a few schools are giving it as a gift to their graduating youngsters.

2.15.20 2:42pm - (Replying to @Ostrov_A) Your tweet has made my day, thanks. And may that, from now on, the inspirational story of Israel will be told through its miracles, not through sacrifice.

2.14.20 2:56pm - Sharing a poem I wrote on the tenth anniversary of Daniel's death: The Lion's De n anniversary-of-his-death/

2.14.20 6:40am - Today we will say Kaddish in memory of our son, Daniel If you happen to be in Encino, CA, join us at 5pm at Valley Beth Shalom. Else, please join us spiritually. After the Kaddish, please sing with us: Tsadik Katamar Yifrach (Like an Immortal Palm Tree)

2.14.20 5:22am - I am surprised that, with all their wisdom and resources, the authors do not consider combining the two, as advocated here: Can anyone inform them that the tools of combination are available. #Bookofwhy

2.14.20 4:07am - For ML folks, "domain adaptation" connotes an insurmountable obstacle. For CI folks it is a causal graphs problem embraced under "transportability" theory. This paper views the problem as Bayes inference on graphical models. #Bookofwhy

2.13.20 2:20pm - Thanks for posting. I was not aware of this paper, which is the most comprehensive I've seen on instrumental inequalities and their extensions. Highly recommended to those who wish to test the assumptions behind instrumental variables #Bookofwhy

2.13.20 12:51pm - It's great to see the principles of causal inference propagate through multiple eyes, multiple lenses and multiple wavelengths. At the end, I believe we will have a super bright sun beam that even blind heathens would notice. #Bookofwhy

2.13.20 6:31am - From my recollection, the Instrumental Inequality is tight for binary treatments, see, and we also have tight bounds for monotonicity , see Thus, if we find violation, its good to know if its structural or functional #Bookofwhy

2.12.20 8:59pm - The insight of Yossi Klein Halevy and his careful choice of words have always swept my admiration. This piece in particular deserves careful attention, it is half prophesy and half calling.

2.12.20 6:40am - A new paper has reached my desk "Revisiting Sequential Attributable fractions" . "Attributable fractions" is an old epidemiological concept that has been treated badly when statistics ruled epi. Interesting to see it in causal dressing. #Bookofwhy

2.12.20 7:23pm - Epi was "ruled by statistics" when it allowed causal concepts such as "attributable fractions" and even "confounding" to be defined in statistical terms, and no one complained. This was "ruling" by language and thought. #Bookofwhy

2.12.20 1:10pm - Sample complexity has been both an enigma and Achilles heel of causal, non-parametric estimation. This paper establishes sample and time complexity bounds on non-parametric identifiability, thus bringing CI closer to the general ML fold. #Bookofwhy.

2.12.20 6:19am - At the request of readers, I am sharing a transcript of my ACF talk on "Two New Weapons for Reclaiming Israel's Posture on Campus" Use them while you can in defense of equity, diversity and inclusion.

2.12.20 12:51am - For those of us still interested in Lord's paradox, and his question: "how to allow for uncontrolled pre-treatment differences between groups", I've added a comment to my post, examining why statisticians have given up the challenge prematurely #Bookofwhy

2.11.20 1:26pm - As Israel celebrates #WomenInScienceDay I can't help recalling how, in 1960, my EE class at the Technion had 4 female students among 120 males (1 of the 4 was my wife). Today the percentage of women is 40% ( 20% of them Arab). A more inviting culture for a male student.

2.11.20 6:26am - I used "head-to-head" in the Bayesian Network paper of 1985 and thereafter. But "collider" became more popular later, and I believe it was coined by the CMU group of Spirtes etal in 1994. #Bookofwhy

2.11.20 6:14am - I wish someone rewrites this wiki entry. I have never heard of a book that is "laying the foundations of the modern debate on causal inference". The writer tried so hard not to take sides that naive readers may take CI to be just one big "debate". #Bookofwhy

2.11.20 5:58am - Warning! Good paper, but I would not recommend expressions like: "A DAG is one method that investigators may use to clarify which variables have causal interpretations in statistical models."1st, NO method can clarify that. 2nd, DAGs are causal, not statistical models. #Bookofwhy

2.11.20 2:17am - It was a pleasure to present the logic of "Emancipating our Identity" and to see how the word Zionophobia gives people the power to express what they have been yearning to express all along. I am now convinced, Zionophobia WILL become the ugliest word on campus.

2.11.20 1:47am - (1/ ) This is one of the most perplexing question in Jewish history: to embrace or to shun. King Solomon said: To every thing there is a season... a time to embrace, and a time to refrain from embracing (Proverbs 3). I feel we are approaching a tipping point in our existence where
2.11.20 1:47am - (2/ ) we should CLOSE RANKS rather than EXTEND TENTS. Those who trade core values (eg, Israel) for social acceptance are becoming existential threats, thus making a painful community split or a unilateral secession almost inevitable.

2.11.20 1:26am - (Replying to @nicidob @maximananyev and @TradeandMoney) Recalling that "what's occurring" in not the data, but the ropes behind the data, "a sketch of what's occurring" is precisely what a causal model gives you, and what MHE pretends to ignore. #Bookofwhy

2.11.20 12:16am - I am not after definitions. I am intrigued by your accurate description of day-to-day practice of social scientists, a confession I could not obtain from practicing social scientists, or from authors of econ. texts. No qualms. #Bookofwhy

2.11.20 11:03pm - I think you have summarized the current practice of "natural experiment" fairly well, though MHE gurus would prefer to dress it up in more socially accepted terms, e.g., Note, matching cannot undo model-blindness. See #Bookofwhy page 274.

2.10.20 10:40pm - I could not let this phrase pass w/o a comment: "a pretty standard regression analysis. It's causal in that the coefficients show predictive power". Now I do not know where to begin. Perhaps here @Bookofwhy? Perhaps Wright (1920)? Perhaps the "microscope"

2.10.20 2:21am - Your tweet has enticed me to share a nostalgic piece from my ancient past. It is a popular exposition of the hopes that superconductivity inspired in the 1960's: . Enjoy. Nothing like #Bookofwhy though.

2.9.20 11:48am - Agree with @eliasbareinboim . This paper gives a more extensive and formal characterization of notions such as autonomy/locality/invariance that are usually attributed to causal relations. #Bookofwhy

2.9.20 6:50am - (Replying to @yudapearl and @attilacsordas) Sorry, I meant do(slippery), not do(wet).

2.9.20 6:48am - It also contains answers to: "Who needs causality if all you want is prediction". As I once tweeted, ask any pollster what the secret is to good prediction and the answer will soaked with causal vocabulary. Why? Integrating data from multiple sources (Fusion) is a causal exercise

2.9.20 6:24am - I watched this Ted talk in 2016, impressed by the capability of her system. Last month she gave an interview at the Technion, Israel, (our alma mater) and, when I watched it again I noticed that this is the best promotion for causal networks in existence (not counting #Bookofwhy)

2.9.20 6:36am - (Replying to @BenjaminShender) You mean conspiracy theories that emanate from hate are genuinely believed by the victim (eg., Roger Waters) and cannot be labeled "dishonesty", despite his rational knowledge of the truth. Interesting. My heart goes out to him.

2.9.20 6:24am - I watched this Ted talk in 2016, impressed by the capability of her system. Last month she gave an interview at the Technion, Israel, (our alma mater) and, when I watched it again I noticed that this is the best promotion for causal networks in existence (not counting #Bookofwhy)

2.9.20 6:05am - The journey to predict the future: Kira Radinsky at TEDxHiriya via @YouTube

2.9.20 3:55am - (Replying to @attilacsordas) Banana peel is a good instantiation of do(Wet), but the DAG does not need such specific external forces. Just tell it do(Wet) and it will imagine one on its own. On page 71 we will indeed represent do(Wet) by a force F(Wet), but why load the DAGs with redundant nodes? #Bookofwhy

2.9.20 2:45am - (Replying to @EpiEllie) Sorry, this tweet was meant for @EpiEllie and the toddler who was so impressed by what he thought he should have known. #Bookofwhy

2.9.20 1:55am - (Replying to @Irishchutzpah) Why give Roger Water the benefit of a "thinking disorder" or "hereditary disease". He knows the truth about Israel and still acts as if he believes the slogans against it. That makes him "dishonest", not "sick".

2.9.20 1:21am - (Replying to @Spinozasrose and @m_pezzolla) It is not "hatred" nor "antisemitism" but a straightforward continuation of the culture of deceit, as conveyed to The Guardian by Ehud Barak The more of it, the more its character is exposed and understood.

2.9.20 12:33am - (Replying to @Claire_Voltaire) Bernie and @ewarren have not learned from Corbyn's defeat, they think those grand "coalitions" represent anyone but their creative slogan writers. Like Corbyn, they underestimate voters' intelligence and voters' sense of right and wrong.

2.9.20 12:21am - (Replying to @PHuenermund) If you read the "linear microscopes", here and here, you will know more than what most Econ&PoliSci will ever know. And that's a lot. @Bookofwhy

2.8.20 11:45pm - Which raises a deep cognitive science question: Can dishonesty be localized? Can an artist be dishonest in his moral makeup and, simultaneously, honest in his artistry?

2.8.20 11:26pm - From my current reading of the article I do not see the super-breakthrough advertised. I see a serious attempt to narrow the set of data-compatible structures, using recently proposed (2006-) asymmetry-producing ASSUMPTIONS. Worth watching & reprimanding the advertiser #Bookofwhy

2.8.20 10:44pm - I cannot overemphasize the importance of page 24 for understanding: "intervention", "parents", "invariance" and other notions, often treated informally. Chapter 3 later explicates the ramifications of this one page, and gives us backdoor, do-calculus and much more, #Bookofwhy

2.8.20 10:04pm - My understanding of "measurement errors" is summarized in two papers on "effect restoration", linked via and I have the hunch the answers can be found there. #Bookofwhy

2.8.20 9:56pm - The advertisement for this super-breakthrough in causal inference is unfortunately too sketchy to comprehend. I hope someone attends the AAAI-2020 talk and tell us, in technical language, how they overcome the First Law: No causes in - No causes out. #Bookofwhy

2.8.20 10:20am - This is something (some) American Jews do not understand (e.g., New Israel Fund, post-Zionist Synagogues) as they accelerate the disappearance of American Jewry by weakening the moral basis for Israel's sovereignty.

2.8.20 9:46am - (Replying to @AngeloDalli) It is surely possible, because we can always fit a function to data. The question is whether the fitted function (w/o causally accounting for daily temperature) would correspond to our understanding of what banning ice cream will do to the world. #Bookofwhy

2.8.20 3:09am - (Replying to @ZionessMovement @JewishCJustice and 2 others) Anyone knows whose Bernie's surrogate is? Who makes up the questions? Whether anyone there is prepared to ask the hard questions? And where we can find the program?

2.7.20 8:20pm - It's final! Eden Alene will represent Israel in the Eurovision contest 2020. When I saw her perform: I could not control my goose pimples. Thinking how her parents walked the vast deserts of Ethiopia to give their children the opportunities they now have.

2.7.20 7:53pm - Thanks for retweeting this exercise. It was proposed in Sep 2019 but, in light of our recent discussion on DL and explainability, it remains doubly relevant. Let's do it together. #Bookofwhy

2.7.20 7:28pm - (Replying to @oacarah @JohannesTextor and @ProfMattFox) Agree, the "microscope" is a useful paper, I frequently consult it when linear models come up. I wonder why the dont teach it in Stat 101, since it can be taught as an innocent exercise in linear regression, not mentioning the C-word. #Bookofwhy

2.7.20 7:16pm - (Replying to @memosisland and @ylecun) Not so. Loops are allowed. See Causality page 315, and its complete solution on page 316. Dont miss footnote 10, on my funny experience with solving a problem that economists themselves posed. #Bookofwhy

2.7.20 8:34am - (Replying to @EpiEllie and @PHuenermund) It is alien in two senses: (1) The founding fathers of econ, Haavelmo, Koopman and Marschak would rather quit academia than teach from MHE. (2)Thinking about an economic reality as an RCT is convoluted and constraining, which is why they can't use DAGs, only IV. #Bookofwhy

2.7.20 8:22am - (Replying to @EpiEllie and @PHuenermund) This is the secret of Neo-econometrics. They quote and quote and requote themselves, which helps create an illusion of progress. But see what this echo-chamber quoting amounts to in my answer to Imbens; #Bookofwhy

2.7.20 8:07am - (Replying to @nano_unanue) It does not need to be a DAG. SCM may have a cyclic structure. But when the structure is acyclic certain properties come into being what facilitate analysis and identification and do-calculus. #Bookofwhy

2.7.20 8:03am - (Replying to @EpiEllie and @PHuenermund) The other way around (remember cause and effect?). I do not like Target Trial framework (do you still call it "framework"?) because I can see which direction it takes, not the other way. #Bookofwhy

2.7.20 7:47am - (Replying to @PHuenermund and @EpiEllie) I am not sure we agreed here, on tweeter, which direction it goes. Remind me, but I am fairly sure it was not the "right" direction. #Bookofwhy

2.7.20 7:15am - (1/ ) It is refreshing to read about "Causal diagrams: pitfalls an tips" by folks who actually use DAGs, as opposed to distant commentators. I recommend however that the paper be read in the context of (1) my critique of Imbens
2.7.20 7:15am - (2/2) and (2) the examples in which deconstruct myths about TWIGs. #Bookofwhy

2.6.20 4:38pm - If you are planning to attend AAAI-20 in NYC dont miss our humble contribution to causal discovery, Monday 2/10 Technical session 10: Reasoning under uncertainty. "For the want of a nail... a victory was won/lost" (BF) #Bookofwhy

2.6.20 8:21am - More on Fairness, this time in Nature accepting and explaining the necessity of causal models to define and quantify what we mean by Fairness or absence of Fairness. @Bookofwhy

2.6.20 7:46am - (Replying to @TheAngrySemite) Mena is where your language incubated, where your heroes sang their songs, and where your mind wanders when you say "homeland".

2.6.20 5:04am - (Replying to @joedotfaith @ylecun and @federalreserve) Very interesting! I wasn't aware of those regs. What backgrounds did the authors have?

2.6.20 2:39am - (1/n) Two comments on @ylecun thread, on doing engineering w/o causal explanations. True, flight engineers can improve wing design with Navier Stokes eq. But naive users need naive explanations on why a loan application was denied, gradient-based optimization in NN will not satisfy
2.6.20 2:39am - (2/ ) this need. I touched on this aspect of man-machine communication here 2nd comment: Every heat engineer knows the 2 laws of thermodynamics, albeit not their atomic explanations. These 2 laws have saved humanity many hours of searching for perpetual
2.6.20 2:39am - (3/ ) motion machines, and many many hours reviewing and rejecting proposals for such machines. In machine learning we also have two conservation laws: 1. Conservation of interventional information (the barrier on Rung 1) and 2. Conservation of counterfactual information (Rung 2
2.6.20 2:39am - (4/4) barrier). Taking seriously the analogy with engineering, one would insist on teaching ML thermodynamics to every ML class in the country. Would Facebook join UCLA in initiating this huge educational project? Just think of the savings in perpetual motion machines. #Bookofwhy

2.5.20 4:23am - (Replying to @shamoons) No! Jihadi music is Jihadi music. "From the river..." is a death chant to human beings living in the neighborhood. Not to mention the students whose meeting was silenced by the saintly "simple chant".

2.5.20 2:02am (Replying to @arxter and @Claire_Voltaire) Cultures void of history claim "indigenous is a silly notion" Others embrace it to heighten creativity.

2.4.20 10:58pm - I've heard this Jihadi music since I was 3 yrs old. What's new is the participation of gullible American students. We, academics, are partly responsible, by begging protection from anti-semitism instead of demanding a stop to anti-Zionism - the more lethal of the two racist cults

2.4.20 3:44pm - I do not give up and claim (as did Kruskal and Lord) that the solution lies beyond statistics. Rather, I am asking: (3.1) What information is needed for a solution? (3.2) What notation would this information be cast in? (3.3) Would I be able to read this notation and judge the
2.4.20 3:44pm - plausibility of the claims? (3.4) What would I do with it, once I agree with the plausibility? (3.5) Does it have any testable implications? (3.6) Unlike my semi-revitalized colleagues, I will begin critiquing specific models only when satisfied with (3.1)-(3.5). #Bookofwhy

2.4.20 2:02pm - The last question is answered fairly well in the conclusion section of the Lord Paradox posting A revitalized mainstreamer recognizes that (1) The two clashing intuitions are deeply entrenched in statistical thinking and should not be brushed off , and
2.4.20 2:02pm - (2) Both intuitions are causal, hence, to reconcile the apparent clash between them we need a causal language; statistics alone won’t do. A fully revitalized mainstreamer goes further: (3) accepting that every causal assertions must invoke untested causal assumptions I do not

2.3.20 5:40am - Replying to @Plinz and @ArcusCoTangens) I would like to believe that defenders of mainstream listen to me, not because I have an authority in any subject, but because my arguments for revitalizing mainstream make good sense and science would benefit from the revitalizatioin. #Bookofwhy

2.3.20 1:05am - Any disdain or conspiratorial undertones on my side are imaginary at best. In fact, as I articulate here: I respectfully invite mainstreamers to join me in the effort, by temporarily halting the question "What if the model was wrong?" and attend to another
2.3.20 1:05am - , equally important question: "Suppose it was right, what would we do with it?". Attending to this question is pre-requisite to resolving causal problems such as Lord's Paradox. (or, more generally: should we adjust for base-line conditions?) #Bookofwhy

2.3.20 12:07am - (Replying to @jtrecenti @athos_damiani and @agpatriota) Perhaps you can explain "the situation" w/o DAGs. But I have not seen an explanation of why we should come to a different decision, with the same data, depending on the story. #Bookofwhy

2.2.20 11:57pm - (Replying to @MarcusCrede) First, the data show that initial weight is correlated with diet. Second it stands to reason that over-weight students would choose a dining room differently than under-weight. #Bookofwhy

2.2.20 9:55pm - (Replying to @ArcusCoTangens) The question was: what information do we need to decide correctly. This can be answered independently of the question "do we have this information?" or "do we have sufficient evidence to support the needed information?" Separating tasks does not mean neglecting tasks #Bookofwhy

2.2.20 8:34pm - Correcting a link to the Lord Paradox posting. The correct link is and it should go to: Lord Paradox and the Power of Causal Thinking. (Thanks to Stephen Leroy for noting). #Bookofwhy

2.2.20 8:25pm - (1/ ) About a dozen or so readers have offered creative proposals for resolving Simpson's paradox in the X,Y,Color scatter plot example. I can't comments on each of the proposals, but I would beg the discussants to focus on my humble proposal: A causal model is both necessary and
2.2.20 8:25pm - (2/ ) sufficient for resolving the paradox, namely, for deciding if X increases Y for a person with unknown color. I would be happy to respond to anyone who thinks this statement is in some way incomplete. #Bookofwhy

2.2.20 8:08pm - (Replying to @jtrecenti @athos_damiani and @agpatriota) What explanation is "equivalent" to DAGs which does not use DAGs? perhaps PO? or "exchangeability"? or "higher resolution?" or "context-sensitive"? I am yet to see one. #Bookofwhy

2.2.20 5:22pm - (Replying to @jtrecenti @athos_damiani and @agpatriota) Why do you say "Causal models are useful" instead of "are necessary"? "Useful" is what economists use to justify not using (see When we have same data demanding two different conclusions depending on the model, we say "necessary" #Bookofwhy

2.2.20 5:08pm - (Replying to @agpatriota @eduardohorta and @michael_nielsen) Explanations are man-made, but we are facing a decision that has true/false value: Will Joe gain a higher Y with more X ? Philosophy aside; do we or dont we have enough information in the scatter plot to decide correctly? #Bookofwhy

2.2.20 4:59pm - (Replying to @DiogoFerrari) I would say: It is impossible to "deal with Simpson's Paradox" without a causal model, and it is impossible to specify a causal model in the language of probability distributions, however intricate. See which you've cited but not taken seriously.#Bookofwhy

2.2.20 4:37pm - (Replying to @eduardohorta @agpatriota and @michael_nielsen) I interpret your hesitation to mean: It depends on the causal relationships between X, Y and Color. Agree. Absent these relationships we cannot decide if X would increase or decrease Y for that person. No need to bemoan science, we need only glance at the causal graph. #Bookofwhy

2.2.20 7:33am - (Replying to @eduardohorta and @michael_nielsen) It surely helps. And the real puzzle begins here: A person with unknown color comes in, would X increase its Y or decrease it? #Bookofwhy

2.1.20 5:49pm - (Replying to @ArcusCoTangens) Too bad for the standard. Graphs are man made. Better to define things in their organic habitat - probabilities.

2.1.20 5:41pm - (Replying to @attilacsordas and @d_spiegel) It is not the "style" that gets things in order, but the introduction of the 3rd-dimension: causation. Hopefully a new chapter in @d_spiegel next edition.

2.1.20 5:31pm - (Replying to @agpatriota) It's the appendix to a prepublication report. It proves that reversal cannot occur in causal logic, in which the notion of "good for men" is expressed in do-operator. And it claims our intuition comes from this calculus, not from statistics, where reversal does occur. #Bookofwhy

2.1.20 11:45am - Cook & Campbell spent years enumerating threats to validity -- they could not do any better, and this paper explains why: Today, that we know how to establish external validity, it is a pleasure to see all the threats that are circumvented. #Bookofwhy

2.1.20 9:55am - (Replying to @agpatriota) No! do(x) does not appear in I just identified where the paradox comes from -- i.e., clash between causal intuition and statistical logic -- and resolved it by replacing stat logic with causal logic. Plain commonsense. #Bookofwhy

2.1.20 8:56am - I do not believe any of these countries thinks this "peace plan" was meant to be a peace plan. It was meant to be a sober reminder to Palestinians that time may no longer be on their side, to stop the tantrum and accept their neighbors as permanent, equally indigenous neighbors.

2.1.20 5:10am - A colleague alerted me to a new wikipedia entry on Market Blanket It is badly written, defining MB as a property of a graph, instead of a probability distribution. The most astonishing feature, uniqueness under positivity, is not mentioned. #Bookofwhy

2.1.20 5:04am - (1/ ) Curious me took a glimpse at Michael Nielsen's blog, which triggered Gelman's discussion of Simpson's paradox Michael is blunt: "[The paradox] shows that some of our ingrained intuitions about statistics are not just wrong, but spectacularly wrong." Yet
2.1.20 5:04am - (2/ ) Yet 90% of living statisticians still believe it can be "resolved" by re-visualizing data. My esteemed discussants on The American Statistician, (eg, Xiao-Li Meng) wouldn't even utter the word "causation". And it is 2020, and Statisticians get angry
2.1.20 5:04am - (3/3) when I remind them of the date, and their students still can't cope with a paradox that has haunted statistics for the past 120 years. Pearl unfairly bashes statistics, they say. #Bookofwhy

2.1.20 4:05am - (Replying to @zakkohane and @StatModeling) I am no longer convinced this is true, after seeing readers comments on "compatibility among the experts". The only way to grasp differences and commonalities is to take ONE toy problem and try to solve it using competing "approaches". The rest is Hollywood. #Bookofwhy

2.1.20 3:56am - (Replying to @dalmiaman) Don't let them "rehabilitate" you back into correlations.

1.31.20 11:04pm - (Replying to @KordingLab @danilobzdok and 2 others) 90% of living statisticians say the same:"I am interested in non-causal questions". Fine! But when a scholar titles a blog "causal inference" you expect to see some interest in causal questions on that blog, not "you do yours and I'll do mine" #Bookofwhy

1.31.20 7:56am - My disagreement with Gelman is fundamental, because his views represent an attitude that paralyzes wide circles of statistical researchers. My initial reaction was posted on Related posts: and #Bookofwhy

1.31.20 7:28am - (Replying to @KordingLab @danilobzdok and 2 others) That's one of them, thanks. As you can see, the causal character of Simpson's paradox is avoided like a plague. Why? I am asking you as one who knows the solution: Is it possible to solve w/o graphs? Those who can't solve a problem, avoid it, and cover up the void..#Bookofwhy

1.31.20 6:13am - (Replying to @danilobzdok @blei_lab and @StatModeling) Does anyone in 2020 believes that linear regr can be used for causal discovery?

1.31.20 6:10am - (Replying to @KordingLab @danilobzdok and 2 others) In person? You must be kidding. If you knew the number of hours I spent on his blog, you would not say "different language". The example that brings everything to the surface is Simpson's paradox. I dont have the strength to dig it again from his blog. If you can, please share.

1.31.20 5:47am - (Replying to @KordingLab @danilobzdok and 2 others) I am not so sure. Why? Because all my attempts to convince @statmodeling to show us how he solves a simple causal problem (one whose solution is known in advance) ended up in failure. I lost my charm. Simpson's would be a good example #Bookofwhy

1.31.20 5:22am - (Replying to @mf_schomaker) One argument is, indeed, that it should have been encoded in the SCM. Another, that it should be treated as "disjuctive action", and extrapolated by imaging: #Bookofwhy

1.31.20 5:11am - (Replying to @sbuhai) But look at the cost of this "standard". Invited papers tell readers: "Here is a body of work that has been neglected, catching up requires that we waive the "originality" requirement." If it were not for such editorial leadership, my work would be unknown outside CS. #Bookofwhy

1.31.20 3:59am - (Replying to @sbuhai) I beg to differ. An "invited survey" has a totally different status than ordinary submission. The requirements of innovation and importance are waived, b/c the editor decides on the latter. There remains only the requirement of "representation." #Bookofwhy

1.31.20 3:37am - Retweeting my rebuttal to Imbens's paper, because readers complained about bad @@ signs in the original post. The link is still the same: #Bookofwhy

1.31.20 1:43am - (1/ ) Another important clarification. In what way does SCM embrace the logic of PO? Ans. It supports the consistency rule Y_x = Y if X=x, which is the main inference engine of PO. Consistency a theorem in SCM and an assumption in PO (part of SUTVA, which
1.31.20 1:43am - (2/ ) deals with side effects in experimental settings). In this sense one can safely say that SCM provides legitimization for the logic of PO, but rejects PO as a "framework" or "approach" #Bookofwhy

1.31.20 1:08am - (Replying to @ThomSeaton) Don't give up so easily. "Ein Lecha Adam SheEin Lo Shaa" says the Mishna (Every person has untapped potential). Even "y=mx+b" is not trivial. I once asked a statistician what Y_x is, when b is correlated with x. Repeating this story would get me into another trouble. #Bookofwhy

1.31.20 12:30am - (1/ ) Important clarification of this point: SCM embraces the counterfactual notation Y_x and its logic, but not the "PO approach" which is a research methodology built around Y_x. The fundamental difference: SCM starts with what you believe (eg DAG), PO starts with what you need (e
1.31.20 12:30am - (2/ ) (eg., ignorability conditions). Why can't PO start with what we believe? Because it insists on expressing everything in the language of Y_x, no structure, no DAGs, while SCM starts with the language in which scientific knowledge is stored: "who listens to whom?"#Bookofwhy

1.30.20 11:47pm - A chapter in history. Readers asked when the relationship between PO and DAGs was first reported. My earliest record is this '93 Stat Conference in Florence (Section 6) As I recall, the audience had NO IDEA what I was talking about - blank eyed.#Bookofwhy

1.30.20 8:27pm - (Replying to @Jabaluck) Can we skip the "unfortunate" - the fault is all mine. And can we start rejoicing what we can do today that we couldn't yesterday? Rejoice man! Show your students how to do it. And write to Imbens to open Econometrica to those who want to learn. #Bookofwhy.

1.30.20 5:18pm - (Replying to @GregZ_MD @ipam_ucla and @GlockerBen) Would have loved to hear the talk, or get the paper, if available.

1.30.20 5:14pm - (Replying to @wagonomics @causalinf and 2 others) Clarification! @yudapearl has embraced the counterfactual notation Y_x, as did PO. But he could not embrace the "PO approach", which permits researchers to assume what they need, instead of what they believe. Not their fault, PO could not express what they believed @Bookofwhy

1.30.20 4:27pm (Replying to @btshapir and @Jabaluck) Research answer: Economist: Citation for me? Citation for whom? Non-economist: Rejoice man! Look what we can do today that we couldn't yesterday! Rejoice!

1.30.20 1:51pm - For the benefit of all readers, I have compiled my "Causal, Casual and Curious" articles in one searchable page, now posted here: . Enjoy. #Bookofwhy

1.30.20 1:38pm - (Replying to @Jabaluck) You are right. I was saying it more excitedly: "Rejoice! ALL (quantitative) Economists should find it useful" . I say the same today: Rejoice!

1.30.20 5:26am - (1/2) It is FAT2020 week, so I'll not spoil the fun. But I am not convinced by Issa's argument that counterfactual logic cannot capture fairness because it presumes autonomy which breaks down in social systems. Heckman made similar arguments for economic systems: "Shutting down one
1.30.20 5:26am - (2/2) equation might also affect the parameters of the other equations in the system and violate the requirements of parameter stability." (Heckman and Vytlacil 2007). I countered here, as well as Cartwright's critique of autonomy, earlier.#Bookofwhy

1.30.20 1:32am - @causalinf , @PHuenermund , @EconBookClub, @jeremyskog Speaking of heathens, converts and imposters in these last days of DAG-deprived economics, I've posted a brief rebuttal to Imbens paper on PO and DAGs: Comments are welcome. #Bookofwhy.

1.30.20 1:06am - (Replying to @attilacsordas) Can you rephrase the question in terms of information relevance, given that we "know". ???

1.29.20 3:37pm - (Replying to @shamoons) The question was addressed to colleagues who are dancing to the eliminationist tune of "from the river to the sea", not to Governments, organizations, or phenomena. I assume you are not one of those dancers.

1.29.20 7:20am - (Replying to @TheShubhanshu and @fatconference) It is still only one page long; is that all you have?

1.29.20 7:14am - (Replying to @causalinf) Remind me what our bonus agreement was. But for the life of me, where did find a "skeptical friend"? In 2020? In economics? You must have been searching really hard. #Bookofwhy

1.29.20 6:25am - (Replying to @TheShubhanshu and @fatconference) Thanks for posting. Does anyone has her compete paper? I saw it someplace, and it seems to challenge the capacity of counterfactual logic to capture "fairness", so I am wondering what logic can capture it. #Bookofwhy

1.29.20 6:08am - (Replying to @EmraniMd) I would be careful before I give a cold-blooded Zionophobe the benefits of hereditary disease.

1.29.20 5:18am - What is Causal Cognition? I thought this article will help me find the answer: But not seeing computational models of the key concepts I feel unable understand the goals and methodology of this intriguing discipline. My weakness. #Bookofwhy.

1.29.20 3:50am - For goodness sake, she really thinks apologies were made for self promotion.

1.28.20 10:35pm - Any peace plan that does not address the elephant in the room (i.e., the religion of "From the river to the sea") will be remembered, not for what it says but for the eulogies in its funeral.

1.28.20 8:36pm - (1/2) I had the thrill of discussing this counterfactual world with @DavidDeutschOxf yesterday. He viewed it as a swift allies victory in WW-II, assisted by a sizable army of dedicated Jewish volunteers from Palestine/Israel. I would add to it: a true Palestine-Israel co-existence
1.28.20 8:36pm - (2/2) un-impeded by traumatic memories of the Arab 1948 attack on the Yishuv, and the massive displacement of Arab population in that attack.

1.28.20 5:51pm - (Replying to @EinatWilf) Palestinian detriments are well documented, but their victories are not widely known. One such "victory" was the entrapment of European Jewry, with the British Navy, and its genocidal consequences. The memory of this successful "victory" makes "From the river..." appear feasible.

1.28.20 4:57pm - Everyone of my esteemed colleagues who is dancing and prancing to the tune of "From the River to the Sea" will have to answer to Lady History when she asks: What have you done for peace?

1.28.20 4:57am - (Replying to @RussInMtl @rlmcelreath and @Lester_Domes) Will follow your lead, as long as end up with the latter.

1.28.20 2:46am - (Replying to @rlmcelreath and @Lester_Domes) Why do we assume that students are "scared" by equations? I do not think this was on our minds in 1999. If it was, perhaps we should go back and sprinkle a few more equations there. "A formula is a baked idea. Words are ideas in the oven." #Bookofwhy page. 335.

1.28.20 1:54am - As some of you heard, UCLA is under investigation by the Education Department on charges of "discrimination and harassment," filed by two Jewish students. Here is a LA Times letter written by another student, explaining the background:

1.27.20 5:52am - I am compelled to retweet your pleadings @RealSarahIdan , partly b/c you grew up across the street from where my wife did, in Baghdad, partly b/c you sang Hatikvah with me, loud and clear, in the SWU Conference, but mainly b/c the Iraqi people deserve freedom we take for granted.

1.27.20 5:10am - Sharing exciting nanotechnology news from the Technion (my alma mater), Prince Charles, and Winston Churchill's Auditorium. The connection: Nostalgia: I was present at the 1958 dedication of this auditorium, and later attended lectures there. #Bookofwhy

1.27.20 1:33am - (Replying to @mohomran) And I thought I was the first to name this on-going pattern of lies as "CULTURAL" (not racial). Where did I get the idea of being first? On my campus, UCLA, where the word "Palestinian" means "always right" "holy of holy" "never to be criticized". Anti-Palestinian??? Come to UCLA

1.27.20 1:14am - (Replying to @mohomran) It is not "acceptable" at all. I just coined this expression today, seeing this culture grow from tiny sprinkles of denying Jewish history, to denying the Holocaust, to blood liable, to denying yesterday's snow. But now, having stained the US Congress, someone has to name it.

1.27.20 12:53am - (Replying to @SachaBaronCohen @EinatWilf and 2 others) @SachaBaronCohen , you are so right, and so is @JGreenblattADL . But don't those who delegitimize Israel also aim to encourage another one? What else do they aim to encourage? And watch how close they are getting to their aim when tolerated as "just something people get wrong".

1.27.20 12:30am - Why does everyone want Rashida to apologize? If she apologizes she would surely repeat the lies in different dressings. Let her stew in her own soup as a monument to Palestinian culture of deceit. Some Palestinians will eventually recognize what she does to their reputation.

1.26.20 8:04pm - Speaking about AI, machine learning, what DL can and cannot do? This analysis of 2800 yr old Hebrew inscription tells it all. Can CI be of help? Perhaps, in case we want to generate the explanations automatically. #Bookofwhy

1.26.20 3:43pm - (Replying to @jeffreywatumull @talyarkoni and 4 others) What humans do, we observe; what NN can't do we can prove. Eg, humans shout: "Impossible" when told of a drug that is good for men, good for women and bad for a person. What would NNs do if given Simpson's data? What would a statistician do? See #Bookofwhy

1.26.20 5:21am - Be careful. Hannan Ashrawi is the Darling of the West. She always tells reporters what they want to hear. E.g., When asked if PA recognizes Israel right to exist, she always says: "Arafat recognized Israel in 1988". None ever asks her: "Do you?" They know she can't say YES.

1.26.20 4:45am - Replying to @attilacsordas) I agree that Mackie's INUS is inadequate and full of inconsistencies. I mentioned them in a Tweet discussion on Rothman's "sufficient cause PIE", but Harvard Epis were so wedded to it that I felt like I'm depriving them of a seat in heaven. See #Bookofwhy

1.26.20 4:17am - (Replying to @DaveBrady72) This discussion intrigued me b/c it challenges the capacity of counterfactuals to define "race", but I could not find the paper. Is it available? It raises the question: Are human capable of ever agreeing on the distinction between "racist" and "non-racist" practices? #Bookofwhy

1.26.20 12:53am - (Replying to @talyarkoni @dileeplearning and 3 others) Should we really cast them as "open empirical questions" when we have mathematical results on what is doable and undoable irrespective of NN architectures ?? Not sure. Empirical claims are subject to interpretation when we get to higher level AI tasks #Bookofwhy

1.26.20 12:41am - We spoke about innate templates that help reasoners do prediction intervertions and counterfactuals. This new paper in Developmental Psychology explores how young children identify "causally relevant variables" #Bookofwhy

1.25.20 10:52pm - Not entirely surprising. A culture rooted in deceit spawns leaders of like character.

1.25.20 9:57pm - (Replying to @HannesMalmberg1 and @HarryDCrane) I think of SCM in econ as reclaiming commonsense in economics (hesitating betwee foreign invasion or home-grown abberation).

1.25.20 7:10pm - The more requests I get to participate in live debates with critics of CI, the more I wish to ask if there is anything missing from the "Dialogue with A Hostile Examiner"( p. 369, that we can learn from a live debate or a staged fist-fight. #Bookofwhy

1.25.20 6:36pm - Nor will I get tired of commending those who recognized the educational power of Primer While educators debate where to start - estimation, correlation, PO, or ML - Primer starts where knowledge resides: Structure. The rest follows organically. #Bookofwhy

1.25.20 2:36pm - Replying to @HannesMalmberg1 and @HarryDCrane) Yet despite common ancestry, the new culture of "natural experimentalists" does view "structural economics", or even mere "structure," as an existential threat. If you think this is an exaggeration, I'll dig out some juicy quotes. #Bookofwhy

1.25.20 7:01am - (Replying to @cecilejanssens) Sorry if I misunderstood. My question was also simple: To decide if a {hypothesis + data} supports a conclusion takes more than "reasoning 101". We need a non-standard logic to do it if the conclusion is causal. Will wait patiently for a written summary when available. #Bookofwhy

1.25.20 6:27am - (Replying to @dr_benner) Hats off!!! Still, the entrapment of 1936-1940 needs to be taught, especially in Rashida Tlaib's school district, in Michigan.

1.25.20 6:17am - (Replying to @trumanfrancis) Great question!!! Textbooks tell us: No way! correlation is all that's needed for prediction. Butask any pollster what the secret is to good prediction and the answer will invoke causal vocabulary. Why? Data Fusion is a causal exercise. See #Bookofwhy

1.25.20 5:55am - The most memorable moment I carry from this somber Holocaust memorial is listening to the President of Germany recite the Hebrew prayer "Shehrchianu" ("to the living") instead of "in memory of the dead". So clearly telling the world what Israel is all about. I wish the hundreds,
1.25.20 5:55am - perhaps thousands of Holocaust Museums around the world will follow his insight and dedicate one exhibit room to celebrate the "to the living" message that Israel sends to the rest of the world.

1.25.20 5:25am - Replying to @cecilejanssens) What kind of logic have you used? My reading into the philosophy of science says that, if the hypothesis is causal, classical logic will not do, and we need some sort of causal logic. Which did you find most useful? #Bookofwhy

1.25.20 5:12am - Replying to @MaartenvSmeden and @trumanfrancis) Agree. It holds for any statistical model, any ML algorithm of any form, regardless how smart, and regardless how big the data. It's hard for ML folks to swallow, but I thought statisticians got over the trauma in 2002, when Denis Lindley confessed to this effect. #Bookofwhy

1.25.20 4:34am - (Replying to @trumanfrancis) Indeed. Logistic regression in itself, without extra-statistical causal assumptions, is as helpless as linear regression. It reveals NOTHING about causality. Please alert the thousands of "logistic regressionists" of this fact; textbooks lure them to think differently. #Bookofwhy

1.25.20 4:19am - (Replying to @btolgao @nntaleb and 2 others) What element of irrational human behavior do you think the introduction to #Bookofwhy should have emphasized more?

1.25.20 4:12am - Same goes for readers who ask "could you please explain what causal inference is about?". Please read: "A Dialogue with a Hostile Examiner", page 369, in The entire miscommunication between statistics and CI is encapsulated here in 3 entertaining pages

1.25.20 3:55am - To readers craving to watch a fist-fight: What's the point? You can learn so much more from solving one toy problem in CI. Still, for those who are incurably blood-thirsty, I got one for you. "A Dialogue with a Hostile Examiner", page 369 here #Bookofwhy

1.25.20 3:30am - (Replying to @BenjaminMenashe @nntaleb and @HarryDCrane) I believe @nntaleb had in mind causal inference in probabilistic setting which, of course, #Bookofwhy and each of my writings is advertising in big megaphones. Not to be confused with "probabilistic causality" as an outdated branch of philosophy. See

1.25.20 2:11am - (Replying to @CasualBrady) Sensitivity analysis with graphical models has not made it to textbooks yet." But see this paper for a solid introduction to the framework. #Bookofwhy

1.25.20 1:55am - (Replying to @CasualBrady and @Peninha_13) Agree. "can respond to" is as good. I happened to like anthropomorphic metaphors, which irritates statisticians, but they dig it better. "can respond to changes in" is also useful, unless you talk to a mathematician, for whom "non-trivial function of" is sufficient. #Bookofwhy

1.25.20 1:14am - (Replying to @HannesMalmberg1 and @HarryDCrane) If natural experimentalists where interested in methodology they would rejoice finding their work perfectly aligned with SCM, enriched with more powerful machinery. Unfortunately, they see "structure" as an existential threat. #Bookofwhy

1.25.20 12:40am - (Replying to @avicenna @joedotfaith and 2 others) Moreover, this assumption is "causal", not statistical, so it cannot be expressed as some restriction on parametric family of distribution functions, as we normally find in the p-value literature. Its an assumption of a new dimension. #Bookofwhy

1.24.20 9:02am - (Replying to @mathalytics @BatteryHorse and @HarryDCrane) No shame in self promotion, especially when institutions expected to promote new ideas prefer to perpetuate old ones. I have done a few tutorials at JSM, but a 1-day course should be more effective. Good luck. #Bookofwhy

1.24.20 8:26pm - (Replying to @joe_shipman @nntaleb and @HarryDCrane) Sorry if I misinterpret you meaning of "meaning", it reminded me how I used to argue with Stats who kept insisting on "SEM has no meaning" or "the only meaning of y=ax+eps is E[Y|x]=ax" [Holland 1988] etc. etc. The influx of Tweets from @nntaleb followers reminded me of old days.

1.24.20 8:16pm - (Replying to @HannesMalmberg1 and @HarryDCrane) The partial derivative definition is a good alternative to do(x). It has obvious problems with discrete x but, more importantly, we do not really have a calculus based on partials that can deliver something like the backdoor/frontdor adjustment formulas. Do we? #Bookofwhy

1.24.20 8:04pm - (Replying to @joe_shipman @nntaleb and @HarryDCrane) CI is orthogonal to the Bayesian vs. Frequentiss debate. See here Why I consider myself only half Bayesian. #Bookofwhy

1.24.20 8:00pm - (Replying to @joe_shipman @nntaleb and @HarryDCrane) No offense, but I can tell you are statistician. How? For my stat colleagues things has "meaning" only if they can be expressed in the language of statistics. Causal questions cannot be expressed thus, and I presume this is what is so frustrating to @nntaleb and @HarryDCrane .

1.24.20 7:51pm - (Replying to @analisereal and @HarryDCrane @analisereal) Thanks for reminding us of this ancient thread. Today I would answer it differently: "Bashing Statistics? I was generous! Count 2020 stat textbooks that mention CI, stat dpts offering CI courses, PhD's who can solve a toy problem in CI (eg, Simpson prdx?) #Bookofwhy

1.24.20 7:03pm - (Replying to @HarryDCrane) You are not missing a thing, if you don't have a causal question to answer. If you have such a question we can examine if you can answer it in some other mathematical framework, alternative to CI. I do not know of any, but am always open to learn. How about Simpson's.#Bookofwhy

1.24.20 6:54pm - (Replying to @pastramimachine and @HarryDCrane) I will never get tired of recommending Primer

1.24.20 6:52pm - I concur with @eddericu recommendation, though it depends on readers background. Some are impatient with Primer's gentleness and prefer Causality with all its proofs. #Bookofwhy

1.24.20 6:49pm - Many readers ask me this question about Primer But publishers are driven by greed, and this is one secret they won't divulge. They want you to buy an uncorrected edition by hiding all marks identifying the corrected one. Future authors, Beware! #Bookofwhy

1.24.20 6:37pm - (Replying to @HannesMalmberg1 and @HarryDCrane) Almost! It is logically equivalent to PO, but infinitely more transparent. Similar to Haavelmo? Yes, see the parallels in As to "econ defn" I have not seen one since Strotz & Wold (1960); it has been buried by the "natural experimental" craze. #Bookofwhy

1.24.20 6:26pm - (Replying to @HarryDCrane) You told me you are curious, not antagonistic, so I am searching for your questions, and all I see are statements about what causality is, and what do(x) is good for. So, what are you curious about? I am curious myself #Bookofwhy

1.24.20 6:15pm - (Replying to @nntaleb and @HarryDCrane) I assume you are asking these questions as a curious observer, and you are not in antagonistic mood, like I've mistaken Harry to be. If so, permit me to ask how familiar you are with the CI literature (it is all in the probabilistic dimension) & I will start from there.#Bookofwhy

1.24.20 6:05pm - (Replying to @RotemEren @ShekatkarSnehal and 2 others) Who is blocking? Who is great? Why block? I thought I mentioned Granger causality in #Bookofwhy, but was not sure. Thanks for quoting.

1.24.20 6:02pm - (Replying to @quantadan @ShekatkarSnehal and 3 others) I dont understand. Who is deleting? who is blocking? who is dismissive? I just got assurance that this thread is motivated by curiosity, not antagonism.

1.24.20 5:49pm - (Replying to @agpatriota @nntaleb and @HarryDCrane) Beautiful example. I use it often to demonstrate the difference between Rung-2 and Rung-3 (see Causality, Section 1.4.4). Glad you brought it up. #Bookofwhy

1.24.20 4:33pm - (1/ ) (Replying to @HarryDCrane) Since you are in such antagonistic mood, I'll answer your question using "CI for the Infidel". You are unhappy with the Economist's article on minimum wage, and you want to expose its weaknesses by pointing out how vulnerable the conclusions are to unwarranted assumptions.
1.24.20 4:44pm - (2/ ) (Replying to @yudapearl and @HarryDCrane) So far you are in good company. But now you want to pick the critiqued assumptions effectively, making sure that they are CRUCIAL, not tangential, to the conclusions. CI logic helps you decide whether a given assumption is necessary (aka sufficient) for the conclusion. Nice!
1.24.20 4:51pm - (3/ ) (Replying to @yudapearl and @HarryDCrane) I say "Nice!" b/c no branch of statistics or classical logic can give you such powerful machinery to criticize papers you don't like. But, to be more constructive, here is another chapter in "CI for the infidel" I called it a "survival kit". #Bookofwhy

1.24.20 4:16pm - Anyone with a determination to read Primer is already "up to the task", and will soon join CI. But make sure you've got the "corrected printing". How? Page ix should cite #Bookofwhy. This is the litmus test.

1.24.20 3:54pm - (1/ ) (Replying to @nntaleb and @HarryDCrane) This one is easy. In 1991, I had a quiet dinner with Clive Granger in Uppsala, Sweden. Between the 2nd and 3rd glass of wine, he confessed to me that he feels embarrassed by the name: "Granger causality", since it has nothing to do with causality, but he can't stop people from
1.24.20 3:59pm - (2/3) (Replying to @yudapearl @nntaleb and @HarryDCrane) using it; they need some way to express what they wish to estimate. I think we should honor him by echoing his understanding. An easy way to see that GC has nothing to do with causality is to look at the defining equations and note that they comprise only conditional
1.24.20 4:04pm - (3/3) (Replying to @yudapearl @nntaleb and @HarryDCrane) probabilities, no do(x) expressions, nor counterfactual terms Y_x. Bingo! We are done! Whenever a concept is defined in terms of a distribution of observable variables it can't be "causal". No causes in - no causes out (N. Cartwright) #Bookofwhy

1.24.20 3:36pm - (Replying to @littlebode) I wrote #Bookofwhy and opened a Twitter account so that each one of our 28K followers should be able to, and excited to create a MOOC on Causality. Are you there?

1.24.20 3:23pm - (Replying to @hubertpaulo and @Peninha_13) The invariance of the axioms under changing definitions is important to mathematics, but the question was why not accept ONE of those definitions and work with it as THE definition?

1.24.20 6:45am - The cover looks very familiar. I hope the translators fixed all our errors. Oh, BTW, we just commissioned a Spanish translation. #Bookofwhy

1.24.20 6:39am - (Replying to @ShekatkarSnehal @HarryDCrane and 2 others) People think I'm kidding when I say that most statistically-trained folks are still unaware of what CI is all about. Glad I have this Twitter post from which to watch the glacial progress of science. #Bookofwhy

1.24.20 6:22am - nother forgotten chapter of history. Glad it sees the light of day. My grandparents were stranded in Poland as a result of this Arab-British-Nazi circle of appeasement. It was "Genocide by entrapment", still not taught in Palestinian schools.

1.24.20 3:41am - Nice example of how SCM can serve as a laboratory to test various interpretations of familiar and colloquially used terms, in this case "incentives". #Bookofwhy

1.24.20 3:13am - (Replying to @Meetasengupta) Hilarious, if it wasn't so true. But don't blame AI. Blame big business for enslaving us to premature AI. #Bookofwhy

1.24.20 3:01am - (Replying to @Peninha_13) Euclid "defined" lines and points in terms of other undefined notions, "part of" "length" "breadth" etc. This reductive definition must stop at some point where we choose the irreducible primitives and declare them "self evident". #Bookofwhy chose "listens to" as primitive.

1.24.20 1:48am - (Replying to @tdietterich @BethCarey12 and @GaryMarcus) Let me explain. Interventional data cannot distinguish between two competing Rung-3 hypotheses. Eg, "No effect" in RCT cannot be distinguished from "treatment kills some and cures others". See gentle introduction here: A true Rung-3 Primer. #Bookofwhy

1.23.20 6:44am - Your dream is my dream. Just signed. #Bookofwhy

1.23.20 6:31am - A new dawn in sensitivity analysis! A systematic approach based on meaningful assumptions. #Bookofwhy

1.23.20 6:16am - The brightest light I see in this somber Holocaust memorial is its taking place in Israel, a country that symbolizes a metamorphosis from tragedy to hope, death to renewal and historical injustice to inspiration for all oppressed minorities

1.23.20 2:52am - Clarification: Although the paper referenced does not mention do-calculus explicitly, explainability criteria are defined in terms of interventions on a structural model M meeting condition C. So readers can see what data+assumptions would enable do-calculus to prove C.#Bookofwhy

1.23.20 2:10am - Explainability is ML's Achilles heel. This paper presents formal criteria & measures for various aspects of explainability. Its use of do-calculus gives readers immediate view of the data and assumptions needed to meet each of the criteria. #Bookofwhy

1.22.20 1:48am - (Replying to @udansk) The issue is, if I am not mistaken, whether DAGs could be taken seriously as computational models of human causal inference, as well as of mental store of scientific knowledge. This is a stronger claim than the correctness of the math. #Bookofwhy

1.22.20 1:06am - (Replying to @kerstingAIML @mitbrainandcog and 4 others) Agree. I used "we" figuratively, referring to the trendy community dominated by DL. I am still under the influence of the DL vs. CI discussion we had here a week ago. #Bookofwhy

1.21.20 11:37pm - Speaking for myself, I've gained a lot of insight about myself by playing around with various computational models. Example, I'd never guess that I had a causal diagram in my mind's eye until realizing that no other representation can account for my swiftness and versatility.

1.21.20 10:49pm - In the 1970-80's, AI was all about "computational models of mental processes". With the advent of ML, many now view AI as "algorithmic replacement of mental processes". We lost something in this transition, b/c the fun part of doing AI is gaining an understanding of ourselves.

1.21.20 2:31pm - (Replying to @ArcusCoTangens @bariweiss) and 3 others Enormous difference! The former is emotional, the latter calculated and genocidal. See

1.21.20 1:50pm - (Replying to @udansk) I said DAGs (SCMs) make causal reasoning POSSIBLE. To refute it, you need to present another mathematical object that allows the derivation of predictions interventions and counterfactuals simultaneously. I am not aware of any. Is anyone? #Bookofwhy .

1.21.20 1:40pm - (Replying to @WiringTheBrain) Do we have computational models of that "deep knowledge", ie, how it is represented in the mind and how it is accessed and transformed into a "causal diagram"? Is it not possible that causal diagram are already the deepest form of knowledge accessible to an agent? #Bookofwhy

1.21.20 1:31pm - (Replying to @andrewthesmart) Disagree. "Computational models" can be emulated on a computer. "familiarity" or "fieldwork", even "design", cannot!

1.21.20 1:27pm - (Replying to @SMBrocklehurst and @WiringTheBrain) The SCM (structural causal model) supports feedback loops (see Causality chapter 7). People use the term DAG because acyclicity gives us many computational advantages. #Bookofwhy

1.21.20 6:15am - (1/ ) Has AI done that? To a limited extent YES. Not through DL, sadly. But if you agree that "understanding" means having a representation that allows you to predict and control a phenomenon then DAGs do a fairly amazing job. Where else have you seen a compact mathematical object
1.21.20 6:15am - (2/2) from which you can derive all four: predictions, redrodictions, interventions and counterfactuals? It's primitive, yes, but it offers us a laboratory in which we can test ideas about understanding. Refinements? By all means! Knitpicking? Don't slow us down, please. #Bookofwhy

1.21.20 3:18am - (Replying to @WiringTheBrain @mendel_random and @causalinf) It is science indeed. But AI (at least me) has endeavored to ask a slightly more ambitious question: What has changed in our mental representation of a phenomenon that makes us feel: "We understand something well enough to define a hypothesis that can be tested.". #Bookofwhy

1.20.20 7:07am - I summarize this discussion thus: Progress in our century amounts to building computational models of mental processes that have escaped scrutiny under slippery terms such as "design" "discovery" "background knowledge" "familiarity" "fieldwork". etc. Saying "DAGs are not enough"
1.20.20 10:25pm - (2/3) produces no progress until one is prepared to propose a mathematical model for what IS enough. The IV criterion I presented is necessary for justification, so keep it in mind and teach it to natural experimentalists who should be jubilant seeing their IV's justified.
1.20.20 10:25pm - (3/3) We should always be open to extensions and refinements, but mystification holds us back. We had enough of that in the 20th century. #Bookofwhy

1.20.20 7:07am - (Replying to @deaneckles @thosjleeper and @aecoppock) What makes a promising candidate differ from an unpromising candidate if not features that would enable it to pass the IV criterion against beliefs about the world? #Bookofwhy

1.20.20 7:03am - (Replying to @ryancbriggs @causalinf and @mendel_random) I dont see any debate here. I see an attempt to demystify what some people describe as a mysterious process of obtaining new domain knowledge and spotting a "natural experiment" in the knowledge obtained. #Bookofwhy

1.20.20 6:55am - (Replying to @thosjleeper @deaneckles and @aecoppock) Sure. And then? Eventually, after forming the new simplified DAG, someone has to say: "Hey I got a new IV, and it's a good one too!" #Bookofwhy

1.20.20 6:51am - (Replying to @causalinf and @ez_angus) Agree. I would only equate DAGS with "theory, model, intuition, and deep institutional knowledge". Is there something in the last four that is missing in the DAG? If there is, lets add it, after deciding of course how DEEP we wish to go. #Bookofwhy

1.20.20 6:43am - (Replying to @causalinf and @ez_angus) All circularities disappear if we equate DAGs with "an encoding of what we know". Difficulties may surface if you can postulate a more refined, or more natural encoding of knowledge than DAGs. I can't, but am open to suggestions. #Bookofwhy

1.20.20 6:30am - (Replying to @thosjleeper and @deaneckles) Talking with people is always enlightening, but I am trying to demystify the process of "finding new variables, arrows etc. etc." and see how it leads us at the end to "Hey, I got a new IV". Reaching such a conclusion requires checking an IV criterion on some model. #Bookofwhy

1.20.20 3:49am - (Replying to @deaneckles) I do not see any disconnect. I see a quest to understand (semi formally) what "fieldwork" gives us that we did not have before, and where a "fieldworker" puts the newly learned variables if not in a some revised model. #Bookofwhy

1.20.20 3:07am - Let us try to give "a situation" a somewhat technical interpretation. You do "fieldwork" for 3 months, 24/7, you come back to your office and reflect: "Have I seen a "situation" that satisfies a definition?" What's in memory are you interrogating with this question? #Bookofwhy

1.20.20 12:43am - To focus the discussion: Z is a good IV if there exists a set S of measured variables such that: (1) S does not separate Z from X, and (2) S separates Z from Y after removing all arrows entering X (Causality p.248). We see that the criterion does not require data. #Bookofwhy

1.19.20 11:57pm - Preparing myself for the #WorldHolocaustForum. My grandparents were murdered in Auschwitz in 1942, my son has not been spared the wrath of this hate, and my colleagues in Israel are still under the threats of Ayatollahs designs and Palestinian deligitimization. Preparing myself.

1.19.20 11:23pm - Replying to @PHuenermund Best news from econ. since Haavelmo.

1.19.20 11:20pm - Replying to @maximananyev But you are unfair to people who can only talk "natural experiments", for whom the whole economy is a huge salad of natural experiments, and he who chooses one and ignores the others is called an "experimentalist". #Bookofwhy

1.19.20 9:13pm - (Replying to @DrMikeH49 @StandWithUs and 5 others) Amazing day indeed. My highlights: 1) Singing Hatikva with Sarah Idan (Miss Iraq 2017) and discussing the psychology of Palestinian Rejectionism with Hussein Aboubakr (Egyptian scholar, CCA)

1.19.20 8:35pm - (Replying to @AdanZBecerra1 @PHuenermund and @laura_tastic) Good detective work, thanks. The thing to do now is to petition Susan R. Bailey, AMA President Elect, to bring up this issue for public discussion. My banner would be: "Censor the claims, not the language. Causes have produced effects Myrs before RCT was invented." #Bookofwhy

1.19.20 5:45pm - (Replying to @yudapearl and @EinatWilf) "Culture or mentality?" Sadly, it holds the key to ME peace: If Palestinian Mentality becomes the center of conversation, chances are the embarrassment will sober up minds and plans. Here is a beginning:

1.19.20 5:23pm - (Replying to @EinatWilf) Agree. That's why Omar Barghouti started his lecture at UCLA (Jan 2014) with "They (Jews) are not a people..." Strange, Edward Said insisted on Arabs' exclusive right to define themselves; his disciples now insist on exclusive right to define others. Is this culture or mentality?

1.19.20 6:35am - (Replying to @EinatWilf) If EU has any obligation at all it is to ask Dr. Saeb Erekat if he ever thought whether Israel's has a right to self-determination. Last I heard from him he said: NEVER! Can a people seek nationhood on its neighbor's tomb?

1.19.20 4:15am - (Replying to @yudapearl and @causalinf) For readers asking how one can spot a natural experiment in a model, the answer is very simple and is given here among the toy problems I presented to economists in 2015. To the best of my understanding, they are still working on it. Are they? #Bookofwhy

1.19.20 3:46am - (Replying to @attilacsordas @sciencescanner and @freesci) Discrepancies between RCT and observational studies usually point to confounding (ie spurious correlation) in the latter. However, certain kind of discrepancies may indicate selection bias in the RCT. See Causality page 294. #Bookofwhy

1.19.20 1:03am - (Replying to @aceyuan) Admitting ignorance and overwhelmed by acronyms: What is BERT?

1.19.20 12:58am - (Replying to @causalinf) I am not an economist, nor social scientist. But if I were one, I would first learn how to spot a natural experiment in a well specified model of the phenomenon. Only then I'll try to apply the method to the mental model I have of the phenomenon, however familiar it is.#Bookofwhy

1.19.20 12:07am - (Replying to @HenMazzig) The antisemites are poor victims of hereditary disease. It's the Zionophobes that are doing it in cold blood, destroying every movement that gives them a nod of approval.

1.17.20 10:15pm - (Replying to @AdanZBecerra1 @Andrew___Baker and 3 others) Are you sure the PO guardians of "well definedness" (say @_MiguelHernan ) would agree with your suggestion (and my position in that if you can find it in the DAG it is automatically "well defined"? #Bookofwhy

1.17.20 9:52pm - (Replying to @Andrew___Baker @PHuenermund and 2 others) The notion of "potential outcome" in the PO framework requires "treatment assignment" w/o which you cannot define Y_x. That is why PO folks are debating whether non-manipulative x (say an earthquake) can have counterfactuals -- the RCT roots are still strong & stifling.#Bookofwhy

1.17.20 9:28pm - (Replying to @Andrew___Baker @PHuenermund and 2 others) Many authors are still using "as if randomized" instead of "unconfounded", and the whole enterprise called "target trial" is based on the idea that if you try to emulate some RCT you would do better than trying to think about the underlying cause-effect relations. #Bookofwhy

1.17.20 6:27pm - (Replying to @IzaTabaro) We are partly responsible for fermenting this aberration, by exposing it ONLY when it crosses into anti-Semitism. Aren't they ugly enough, and racist enough, just being anti-Zionists?

1.17.20 6:07pm - (Replying to @PHuenermund @laura_tastic) , Where is this page proof taken from?

1.17.20 6:05pm - (Replying to @christopherruhm @PHuenermund and @causalinf) RCT draws its legitimacy from a proof that, if conducted properly, it delivers a sought-after quantity called "causal effect". This means that causal language must exist BEFORE RCT. The #Bookofwhy makes this point on pages 146-150.

1.17.20 12:02pm - (Replying to @IntuitMachine and @jm_alexia) Examples?

1.17.20 11:55am - (Replying to @MaxALittle and @jm_alexia) Who said it is hard? Only those who have not tried it. Those who have, can’t understand the hesitation of the others. #Bookofwhy

1.17.20 4:58am - Readers who, like me, believe that symbols make history should understand why i felt compelled to retweet this photo. One of the few hopeful signs in the past week, after Omar’s congress refused to support iranian protestors, another history shaping symbol.

1.17.20 4:25am - (Replying to @DKedmey) “listening” is a good and accurate NL metaphor, and so is “source of variation”. But these are good for pre-scientific discourse, or for education. Are we in a similar pressing need for NL definition of “correlation”? #Bookofwhy

1.16.20 5:32pm - (Replying to @PhilosopherMD1) Keep up informed either way. Its been a long time since we heard from philosophers of statistics.

1.16.20 5:27pm - (Replying to @PHuenermund @Lester_Domes and 6 others) The question was not about path-specific effects (for which we have the Avin etal result) but about the Total Effect, ie, E[Stroke|do(Age)]. The miracle of DAGs (not widely acknowledged) is that expressions of type E(Y|do(x), z)] do not need summation over paths.#Bookofwhy

1.16.20 12:40am - Do we need a NL definition of a “cause”? We are blessed with a compact and meaningful mathematical object from which we can derive NL utterances on all three Rungs, association, intervention and counterfactuals! Why chase the unreachable? #Bookofwhy

1.15.20 11:30pm - (Replying to @Lester_Domes @RUBENSARO and 6 others) If we insist on doing path tracing Wright style then, yes, all paths from AGE to STROKE should be counted. Fortunately, we have DAGS to save us: The effect is simply E[STROKE|AGE] , no paths, no multiplication, no additions. Welcome to the magic world of DAGs-land #Bookofwhy

1.15.20 4:58am - (Replying to @PHuenermund and @Andrew___Baker) Indeed, what’s in “design”? Pressing hard, l’ve found that “design” is a word used by economists to cover decisions they prefer to do informally, away from piers scrutiny, thinking old-fashionally that they cannot be made formally. #Bookofwhy

1.15.20 3:47am - Highly recommended watch. And while watching, worth keeping in mind what #Bookofwhy says about Kahaneman’s dictum, and his “undoing project”.

1.14.20 2:48pm - (Replying to @JDHaltigan) If you can be specific, I'll be glad to demonstrate "no malice" in #Bookofwhy

1.13.20 11:48pm - (Replying to @Physical_Prep and @Alan_Couzens) I have great faith in AI, and I have read #Bookofwhy long time ago. The book, too, has great faith in AI.

1.13.20 11:26pm - (Replying to @GodfreySnorgyrs) Why dont you write one for anthropology? Epidemiology owes a lot to that pioneering article, and I am sure anthropology will be grateful to you, if you write one. #Bookofwhy

1.13.20 11:11pm - (Replying to @yudapearl @ShalitUri and @omaclaren) expect a "methodology" to deliver, not guesswork about the dark side of the moon, which we can generate absent a "methodology", still consistent with the standard mathematical definition of "solution. #Bookofwhy

1.13.20 11:06pm - (Replying to @ShalitUri and @omaclaren) That's exactly my point. Before we submit our causal questions to the mercy of DL we need to ask whether we possess sufficient knowledge to prevent contradictory solutions, or in prediction, for which curve-fitting has unique solution. This notion of "unique solution" is what we.

1.13.20 10:51pm - (Replying to @richard_landes @DKedmey and @EinatWilf) I am thinking of fake feminists like Linda Sarsour who is probably sincere in her admiration of Jews like Sen. Sanders, and fake liberals like Corbyn who hate Jews because they support Israel more than he hate them for rejecting Jesus.

1.13.20 9:49pm - (Replying to @omaclaren) A "solution" and a contradiction:
X,Y,Z = 1,2,2
X,Y,Z = 0,3,2
You just told me X=1, now you telling me X=0, and you want me to call it a "solution"? Are DL folks willing to accept both: "the drug is helpful" and "the drug is dangerous" as a legitimate "solution" of DL methodology

1.13.20 9:05pm - (Replying to @richard_landes @DKedmey and @EinatWilf) Your video is very convincing, though I've been exploring another theory, that antisemites like Richard Wagner and Mel Gibson can be forgiven for sucking it with their mother milk, not so Zionophobes, like BDS activists, for whom Israel elimination is cold and calculated agenda.

1.13.20 8:31pm - (Replying to @DKedmey and @EinatWilf) I was not familiar with @DavidDeutsch pattern, probably because I grew up in Israel, with a firm conviction that all past patterns are a thing of the past -- the New Jew will put an end to irrationality and inhumanity. It is 2020, and I am anxious to study the "pattern". Thanks.

1.13.20 8:21pm - (Replying to @DonBeham @RandomlyWalking and @raamana_) Yes, the diagram imposes constraints on the data, so data can refute or satisfy a diagram. This the 2nd fundamental law of causal inference which, fortunately, ML folks are beginning to internalize (unlike economists) .#Bookofwhy

1.13.20 1:13am - (Replying to @TheLeanAcademic @tdietterich and 2 others) The classical (and easiest) case is d-separation; it tells us if some members impose conditional independence constraint on the data. But speaking of Rung-3, things are more involved, and Section 8.5.2 of Causality (2009) has beautiful examples how data shape priors. #Bookofwhy

1.13.20 12:48am - (Replying to @tdietterich @BethCarey12 and @RaajeevVerma) The right way to think about it is, first, to ask if the "evidence" is capable of ruling out members of the model space. The hardest thing for statisticians to swallow is that data can be equally compatible with each member of the space. So, we must act as if we've NO evidence.

1.13.20 12:34am - (Replying to @nitalon and @ylecun) Chibuk L'Katin

1.13.20 12:31am - (Replying to @InnovatArt and @ylecun) This still begs the question: "mathematical models of WHAT?", and what do we do with it once we have it? #Bookofwhy

1.13.20 12:25am - (Replying to @SinghalApoorve) I sometimes feel I need to re-read my own fables to reinforce basic truth. That's what months in the trenches do to you. #Bookofwhy

1.12.20 10:25pm - (Replying to @ylecun) I am the last person to doubt your definition of DL, but I'll have very hard time explaining it to my students; they will rebuke me if I ever say "construct a model" without saying a model of WHAT. Is it a model of the dark side of the moon? Any data from that side? #Bookofwhy

1.12.20 10:05pm - (Replying to @roydanroy @suchisaria and 2 others) Last time (1990) I examined Dempster-Shafer theory, it couldn't even do Monty Hall correctly. See and Anything new to consider? #Bookofwhy

1.12.20 9:54pm - (Replying to @roydanroy @suchisaria and 2 others) We, in the trenches, need more gentle instructions on how you implement SEM/SCM as POMDP when you dont have the state transitions, not even actions, just passive observations. Have you tried it? Has anyone you personally know? #Bookofwhy

1.12.20 9:46pm - (Replying to @InfoSymmetries) Explanation: This is Amazon announcement of the paperback edition, which our publisher decided to release in August 2020. Good news, it will contain all the errata that readers labored to find. Thanks. #Bookofwhy

1.12.20 9:39pm - (Replying to @BethCarey12 @RaajeevVerma and @tdietterich) In my little corner of the universe the answer is: no amount of data can lift us from Rung-1 to Rung-2, or from Rung-2 to Rung-3, no matter how smart we are, and no matter if we call it DL or NN or AI or stochastic optimization. #Bookofwhy

1.12.20 4:44pm - (Replying to @roydanroy @suchisaria and 2 others)

1.12.20 4:44pm - (Replying to @roydanroy @suchisaria and 2 others) Please walk us by the hand through some simplified issues. Take us with POMDP to decide which drug is better, the one that is good for men and good for women, or the one that is good for people. Let's start with the model-free brethern. #Bookofwhy

1.12.20 4:33pm - (Replying to @suchisaria @RandomlyWalking and 2 others) How do we provide them with more talk space than twitter with occasional encouragement to "Please rebel!"

1.12.20 3:29pm - (Replying to @simonbchen) The implicit assumption you are talking about, has hardly been touched in DL circles, which is an interesting social phenomenon: A whole civilization discussing how to get a better fit to data, rather than "what can we do once we get a perfect fit" #Bookofwh

1.12.20 3:16pm - (Replying to @VeronicaEpi) I've not met anyone saying "causal inference is still confusing" after reading #Bookofwhy How can anything be confusing if it merely reflects human thought?

1.12.20 3:08pm - (Replying to @suchisaria @RandomlyWalking and @raamana_) I like your distinctions, for they are cast in terms of the questions we aim to answer, rather than the mathematics used to answer them. I hope they bring DL students to realize: "Gee! We never dealt with questions which we could not answer! The world IS 3-dimensional" @Bookofwhy

1.12.20 2:56pm - (Replying to @RandomlyWalking and @raamana_) "Snark" is not in my vocabulary. The high-school algebra example simply demonstrates that, if we do not have the data to answer certain questions, we should not just guess at random. It has nothing to do with inductive bias, which is a Rung-1 (function-fitting) concept #Bookofwhy

1.12.20 7:03am - (Replying to @davidcnorrismd @venkmurthy and 5 others) I would not call it "maligns". It is a very accurate description that most authors agree with, and only few dare share, which helps perpetuate the mystery. #Bookofwhy

1.12.20 1:50am - (1/ ) (Replying to @raamana_) JP TWEETS (updated 1.13.2020)

1.13.20 1:13am - (Replying to @TheLeanAcademic @tdietterich and 2 others) The classical (and easiest) case is d-separation; it tells us if some members impose conditional independence constraint on the data. But speaking of Rung-3, things are more involved, and Section 8.5.2 of Causality (2009) has beautiful examples how data shape priors. #Bookofwhy

1.13.20 12:48am - (Replying to @tdietterich @BethCarey12 and @RaajeevVerma) The right way to think about it is, first, to ask if the "evidence" is capable of ruling out members of the model space. The hardest thing for statisticians to swallow is that data can be equally compatible with each member of the space. So, we must act as if we've NO evidence.

1.13.20 12:34am - (Replying to @nitalon and @ylecun) Chibuk L'Katin

1.13.20 12:31am - (Replying to @InnovatArt and @ylecun) This still begs the question: "mathematical models of WHAT?", and what do we do with it once we have it? #Bookofwhy

1.13.20 12:25am - (Replying to @SinghalApoorve) I sometimes feel I need to re-read my own fables to reinforce basic truth. That's what months in the trenches do to you. #Bookofwhy

1.12.20 10:25pm - (Replying to @ylecun) I am the last person to doubt your definition of DL, but I'll have very hard time explaining it to my students; they will rebuke me if I ever say "construct a model" without saying a model of WHAT. Is it a model of the dark side of the moon? Any data from that side? #Bookofwhy

1.12.20 10:05pm - (Replying to @roydanroy @suchisaria and 2 others) Last time (1990) I examined Dempster-Shafer theory, it couldn't even do Monty Hall correctly. See and Anything new to consider? #Bookofwhy

1.12.20 9:54pm - (Replying to @roydanroy @suchisaria and 2 others) We, in the trenches, need more gentle instructions on how you implement SEM/SCM as POMDP when you dont have the state transitions, not even actions, just passive observations. Have you tried it? Has anyone you personally know? #Bookofwhy

1.12.20 9:46pm - (Replying to @InfoSymmetries) Explanation: This is Amazon announcement of the paperback edition, which our publisher decided to release in August 2020. Good news, it will contain all the errata that readers labored to find. Thanks. #Bookofwhy

1.12.20 9:39pm - (Replying to @BethCarey12 @RaajeevVerma and @tdietterich) In my little corner of the universe the answer is: no amount of data can lift us from Rung-1 to Rung-2, or from Rung-2 to Rung-3, no matter how smart we are, and no matter if we call it DL or NN or AI or stochastic optimization. #Bookofwhy

1.12.20 4:44pm - (Replying to @roydanroy @suchisaria and 2 others) Please walk us by the hand through some simplified issues. Take us with POMDP to decide which drug is better, the one that is good for men and good for women, or the one that is good for people. Let's start with the model-free brethern. #Bookofwhy

1.12.20 4:33pm - (Replying to @suchisaria @RandomlyWalking and 2 others) How do we provide them with more talk space than twitter with occasional encouragement to "Please rebel!"

1.12.20 3:29pm - (Replying to @simonbchen) The implicit assumption you are talking about, has hardly been touched in DL circles, which is an interesting social phenomenon: A whole civilization discussing how to get a better fit to data, rather than "what can we do once we get a perfect fit" #Bookofwh

1.12.20 3:16pm - (Replying to @VeronicaEpi) I've not met anyone saying "causal inference is still confusing" after reading #Bookofwhy How can anything be confusing if it merely reflects human thought?

1.12.20 3:08pm - (Replying to @suchisaria @RandomlyWalking and @raamana_) I like your distinctions, for they are cast in terms of the questions we aim to answer, rather than the mathematics used to answer them. I hope they bring DL students to realize: "Gee! We never dealt with questions which we could not answer! The world IS 3-dimensional" @Bookofwhy

1.12.20 2:56pm - (Replying to @RandomlyWalking and @raamana_) "Snark" is not in my vocabulary. The high-school algebra example simply demonstrates that, if we do not have the data to answer certain questions, we should not just guess at random. It has nothing to do with inductive bias, which is a Rung-1 (function-fitting) concept #Bookofwhy

1.12.20 7:03am - (Replying to @davidcnorrismd @venkmurthy and 5 others) I would not call it "maligns". It is a very accurate description that most authors agree with, and only few dare share, which helps perpetuate the mystery. #Bookofwhy

1.12.20 1:50am - (1/ ) (Replying to @raamana_) Hilarious! But the more we listen to DL talks, the more it sounds like "everything". I am still hoping one of the faithfuls would remember how the idea that we need n equations to solve for n unknowns changed how we did algebra in high-school. We first asked ourselves:
1.12.20 2:02am - (2/ ) (Replying to @yudapearl and @raamana_) "Do we have n equations?" If not: "Have we forgotten one?", if not: "Perhaps we can impose one?" and then, only then, we submitted our equations to the algebraic machinery that we could trust. The analogous process for CI is spelled out in #Bookofwhy

1.11.20 10:59pm - (1/ ) (Replying to @raamana_) What I did learn from the discussion of "What is DL" is that, though no definition is in sight, DL folks are utterly intoxicated by the seemingly unlimited potentials of their programs, techniques and vocabulary. I remember our second or third week of high-school algebra,
1.11.20 10:59pm - (2/ ) (Replying to @raamana_) utterly intoxicated by its unlimited solution-finding capabilities, the teacher told us that you can't solve two equations with three unknowns. Humbled and disappointed, we were nevertheless excited by this revelation; it saved us hours upon hours of chasing after solutions
1.11.20 10:59pm - (3/3) (Replying to @raamana_) that do not exist. But these were high-school days. #Bookofwhy

1.11.20 2:34am - The intense discussion on "What is DL" has evidently not converged on a consensus among DL practitioners. Nor has it convinced me to reconsider any of the impediments reported in or the ML/CI division of labor recommended. Ready to be educated. #Bookofwhy

1.11.20 1:27am - (Replying to @DKedmey and @tangled_zans) Reply to your tweet removed: @EinatWilf is my hero. I wish Israeli leaders will be as clear as she is on the kind of peace Israel society strives to achieve and what kind of "peace talks" are a waist of time, because they do not entail "equally legitimate and equally indigenous".

1.10.20 4:58am - (Replying to @tyrell_turing) Not really. The other reason for the Q "what is DL" was that good ppl identified DL with everything that AI can ever hope to achieve, which naturally led to the question: "What is NOT DL?" #Bookofwhy

1.10.20 4:54am - (Replying to @witbrock @zaffama and @ylecun) The problem is not what classes of computable functions can be effectively learned by a DL system? But rather, whether the answer you expect your algorithm to deliver is a computable function of your data. #Bookofwhy

1.10.20 4:41am - I am re-tweeting with the hope of reaching one or two econ. students and enticing them to ask some hard questions. #Bookofwhy

1.10.20 4:02am - (Replying to @csilviavr) Your paper makes the point very clear. I was first hesitant seeing the term Causal Bayesian Network CBN, which is defined on interventional distributions (Causality p. 24). But you corrected for it through path-specific, ie. counterfactual notation. #Bookofwhy

1.10.20 1:50am - (Replying to @javisamo) I couldn't access Lily Hu's paper. Can you summarize the limitations of counterfactual reasoning? A punchy example, perhaps? #Bookofwhy

1.10.20 1:41am - (Replying to @PHuenermund @causalinf and @Dunkin_Donuts_2) I think viewing these anecdotes as "personality cult culture" misses the real crisis of econ -- the only data-driven discipline the majority of whose students are still deprived of 21st-century tools such as graphical models, and are not rebelling, not even striking. #Bookofwhy

1.10.20 1:01am - I'm reading this extensive review of Fairness in Machine Learning and am happy to see that it acknowledges (though not strongly enough imo) that "fairness" is a causal notion, and that DAGs plus counterfactuals are needed to make sense of it. #Bookofwhy

1.9.20 7:28pm - As a Special Anointer of saints, I hereby commit the next sainthood to the Editor of the first econometric journal to invite a survey paper on "graphical models in econometric". Spread the word. Candidates may include previous saints. #Bookofwhy

1.9.20 7:03pm - The discussion on "What is DL?" reminds me of identical discussion (among Bayesians) on "Who is a Bayesian?" ending up with a dead-end: "Do it! Dont ask!". Good for justifying what you have been doing, not so good for science, as I confess here #Bookofwhy

1.9.20 3:27am - (Replying to @AndrewLBeam and @ylecun) The division of labor between CI and DL is mathematically defined by every CI exercise. See Each probabilistic expression that appears in the derived CI estimand calls for a DL exercise. Data Fusion entails a similar philosophy

1.9.20 2:33am - (Replying to @_mb46_ and @ylecun) Your simple definition of DL "using ANNs to do stuff" is too broad for me. Is a computer chip an ANN? It is certainly a "network" of wires and gates. What does it take to qualify as "neural" once we allow "artificial". #Bookofwhy

1.9.20 2:27am - (Replying to @tangled_zans) What for you may seem "Politics" is for me "moral imperative". My take: I am a student of Middle East history and a long-time peace activist towards "two equally legitimate states, for two equally indigenous peoples".

1.9.20 2:06am - (Replying to @tangled_zans) The silencing takes the form of protecting this woman from criticism, for fear of being called "Islamophobe". I feel I have earned credibility as a fighter against Islamophobia to articulate what others see but are afraid to condemn.

1.9.20 1:46am - (Replying to @_mb46_ and @ylecun) Surely "considerable differences" exist. But are they judged to be fundamental, or merely temporary limitations of current DL techniques? If the former, then we need indeed to better understand what the the definition is of DL. #Bookofwhy

1.9.20 1:31am - (Replying to @tangled_zans) Random strangers have immense influence on me, but I can't understand what you see wrong in me alerting readers to the danger that this woman poses, through her hate speeches. It is my responsibility to share my analysis with readers who, silenced by PC, do not dare speak out.

1.9.20 12:58am - (Replying to @zaffama and @ylecun) I am not sure DL advocates would be satisfied with your modest definition. They would probably wish to extend it to include "every success story in the next century". But I should not speak for them. Curious: "What is NOT DL?" #Bookofwhy

1.9.20 12:47am - (Replying to @_mb46_ and @ylecun) I believe it was #tdietterich who defined DL as "the science of intelligence systems". And the answer to "Are ANN enough" is trivial: "Of course! Because we know that Organic NN are enough." I am waiting for a less ambitious clarification of what DL is.

1.8.20 11:56pm - (Replying to @tangled_zans) The "bigger picture": To assure descent readers that I share their disgust with the racism of certain Zionophobes, that the educational origin of this aberration is explained, and that it is OK to speak against it. No INSULT here, since Zionophobes are proud of their disorder.

1.8.20 11:32pm - I wish I could comment on the question "Is DL enough?" But I can't, because we do not know what DL is. Some define it as an aspiration to emulate intelligence. Others, eg @ylecun , define it as a specific NN implementation. We are waiting for clarification. #Bookofwhy

1.8.20 11:10pm - Asking a Zionophobe for logical consistency is like asking a snake to walk on two. According to Mogadishu-born Ayan Hirsi Ali, the only semblance of consistency in her (& Omar's) school curriculum in Mogadishu was the consistent blaming of Jews for bad grades and water shortage.

1.8.20 10:46pm - (Replying to @sherrirose and @RhubbBstat) Muchos Congratulationes. #Bookofwhy

1.8.20 10:40pm - OOPPS. I've just answered it here:

1.8.20 10:24pm - It's a bit more that "offsetting paths". Consider Y=f(X,U)= 1 iff X=U, and let X and U be two fair coins. No offsetting paths in X-->Y<--U. Yet X||Y and Y||U. Statistical magic. #Bookofwhy

1.8.20 12:28am - Just bumped into this wonderful Ted Talk of @harari_yuval from whom I've learned about the origins of counterfactuals and imagination, and where I've found a connection between the structural theory of counterfactuals and futuristic robotics. #Bookofwhy

1.7.20 11:00pm - (Replying to @EpiEllie @ruschenpohler and @AndrewJDBell) I would dare say that to ensure generalizability under changing location or time we need to postulate a data-generating-model of some kind. I don't think we can get by with frequency information alone. #Bookofwhy

1.7.20 7:26am - (Replying to @carlgieringer) And how is opposing "killing frogs" a victory for commonsense? The charge varies from year to year, and is becoming more "benign" and "universal". Who does not love frogs? BDS scores it victory from the music "Israel is on trial" and the megaphone, not the libretto.

1.7.20 6:55am - Am. Hist. Assoc. scored victory for commonsense, but the BDS circus will continue. Next year the resolution will call on Israel to stop killing frogs on Tuesdays. Do they? Questions later! First listen to the sweet music of "Israel killing" - BDS symphony #99 in E-flat major.

1.7.20 4:44am - Clarification. MR, IV, and conditional-IV methods belong to bullet 5 (More elaborate policy evaluation.) These are available in DAGGITY, if I recall correctly. It is important to emphasize that DAGs should be used to bestow defensibility upon these methods. #Bookofwhy

1.7.20 12:26am - (1/ ) Comments on "Use of DAGs". Commending you on an important survey. I was worried that DAGs will be introduced too narrowly, but the route of "data generating mechanism" avoids misconception that gets Epi people into trouble when talking "treatment-assignment" jargon.
1.7.20 12:26am - (2/ ) Readers noticed that the papers selection criterion was not clearly defined. E.g., did you include papers where DAGs were used merely to communicate context? or was the causal component necessary?
I would suggest the following taxonomy of DAG usages:
1.7.20 12:26am - (3/ )
1. Communicate contextual information
2. Communicate and defend causal assumptions
3. Selection of covariates for adjustment
4. identify testable implications.
5. Elaborate policy evaluatioin
6. Transportability and selection bias.
7. Missing data
1.7.20 12:26am - (4/4) I was surprised to find no papers in bullets 4-7 Dont people ever test their models? Is Epi still in the adjustment era ? Much work for educators.
Overall, a useful panoramic view of a field gone scientific. #Bookofwhy

1.6.20 6:29pm - (1/ ) I can't wait to see the faces of our two smearing experts when they find that the person whom they labeled "conservative" "religious" "blinded by hatred" etc. etc. is in fact a (1) devout liberal, (2) a voting registered Democrat, (3) a devout atheist, (4)award winner for
1.6.20 6:29pm - (2/ ) anti-hate programs.(5) active writer on Middle East history. What these smear peddlers could not stomach was my criticism of the untouchable saintly Rashida Tlaib. I therefore repeat what I know about her. She has been spewing hate, deceit and outright racism since being
1.6.20 6:29pm - (3/ ) elected. For readers who respect my sincerity, domain knowledge and ability to read beneath the surface, I have also commented on her sweet anti-war statement after soleimani death, and noted that she cannot afford to tell her audience what kind of mass-murderer Suleimani was
1.6.20 6:29pm - (4/4) because a large segment of her constituency sympathizes with Suleimani's agenda of weakening the US and wiping out Israel -- two holy aims that justify all means.

1.6.20 7:25am - (Replying to @timminglab) Moreover, the more you get to know colleagues at Ivy league universities, the more you appreciate the tribal games you did not have to play being yourself.

1.6.20 7:15am - Or, at the very least, @SenSanders , stand up like a Mench and answer questions about what chunks of Sarsour's ideology you DON'T share. I am willing to do the interview, since none of the reporters behind the microphones is willing to touch on this issue.

1.6.20 5:45am - (Replying to @KordingLab) Thanks for the update. Interesting interpretation. The sequence of presenting a method first and asking for veracity second is what triggered my simplistic interpretation.

1.6.20 4:51am - (Replying to @KordingLab) Is there a new "Athey tutorial" in circulation? The last I have seen said essentially: "Whatever economists do is automatically causal" (my humble interpretation). Is there a re-freshener? #Bookofwhy

1.6.20 1:50am - New sounds: "naive, ignorant, unschooled, pasty". What makes some academics so allergic to new information from a new perspective? Put differently, what makes Rashida Tlaib more knowledgeable, educated, revered or morally compelling than humble me? Ready for the first test? is automatically causal" (my humble interpretation). Is there a re-freshener? #Bookofwhy

1.5.20 9:13pm - (Replying to @KordingLab) Agree! Most of the discussed issues are rung-1. Still, they all eventually lead to questions of policy and/or explanation, and the paper gives the impression that DL is well prepared to tackle these issues too. Your honest opinion: Is it? #Bookofwhy

1.5.20 8:32pm - (1/ ) Fair question! My aim was not to "interpret" Rashida's statement, but to express my reaction to her rhetoric in light of what she has been doing in the past. Specifically, in light of the hate, deceit and outright racism she has been spewing since being elected.
1.5.20 8:32pm - (2/ ) 3/3 which do not tell us who Suleimani was and why Rashida can't speak about it. She can't because she would lose her support base, a big chunk of which sympathizes with Suleimani's agenda of weakening America and wiping out Israel. Sad. But rhetoric won't change reality.
1.5.20 8:32pm - (3/3) which do not tell us who Suleimani was and why Rashida can't speak about it. She can't because she would lose her support base, a big chunk of which sympathizes with Suleimani's agenda of weakening America and wiping out Israel. Sad. But rhetoric won't change reality.

1.5.20 8:01pm - (Replying to @bariweiss) Kol Hakavod

1.5.20 6:21pm - (Replying to @fayyazhere)
1. The limitations of rung-3 reasoning are that it relies on information about the functions behind the arrows. Lacking it, gives us bounds, not point estimates.
2. It is useful to assume that my ML colleagues can extract the optimal predictive information available.#Bookofwhy

1.5.20 8:29pm - Beautiful 1-pager. Now I see my math teacher winking to me: Should I be pedantic and spoil the beauty? No. I'll let it go. Just a tiny spoiler: All "statistically dependents" in bullet 5 should read "likely statistically dependents". Why? Some other time. #Bookofwhy

1.5.20 4:52am - (Replying to @ktmud) What you are saying is: It's TOUGH to see someone who has spent lots of time studying the middle east conflict, and worked day and night spreading love and understanding for whole mankind express ideas that differ than MINE. So I'll dismiss him as "blinded by born identity".EASY!

1.5.20 3:48am - This paper encourages ML folks to tackle problems of Climate Change, yet neglects to mention the unique causal and counterfactual tools needed for the challenge. In particular the evaluation of sufficient and necessary causes, eg:

1.5.20 3:27am - (Replying to @TheDavidSJ) If I were the murderer of Rina Shernav (17) I'd say: "Job worth repeating! Even when we kill a girl in the heart of Tel Aviv, we now have someone in the US Congress to blame "Israeli occupation" for the crime. It's a new dawn!" [Note how she did not say "equal rights to Israelis"

1.5.20 1:02am - (Replying to @TheDavidSJ) If she truly cares for "loss of innocent lives", including those hundreds of thousands of innocent lives destroyed by Suleimani, Congresswoman Rashida Tlaib has had ample opportunities to weave it into her speeches since his departure. She has not done so & we know why. She cant!

1.4.20 11:41pm - Congresswoman Rashida Tlaib "cannot stay silent". She is deeply shaken by the unexpected loss of an ideological idol, mass murderer Qasem Suleimani. Her sudden interest in "innocent home and across the globe" now humors her American voters in Michigan.

1.4.20 8:55pm - This is one of the most futuristic, yet sensible futuristic conversations I have heard: I'm only surprised they treat consciousness as a mysterious inaccessible concept, laden with pain and suffering. #Bookofwhy

1.4.20 4:30pm - Glad we agree. I believe, however, that it goes deeper than "origin". The refusal of the PO community (including Econs and Harvard) to adopt the intervention/counterfactual distinction represents refusal to accept structural models as a basis of scientific thinking.#Bookofwhy

1.4.20 3:29pm - (Replying to @jrgptrs @3ieNews and 13 others) If I were able to contribute to your blog, I would ask: "Why quote Deaton who laments careless handling of #externalvalidity instead of promoting methods that properly handle the problem?" For example: or #Bookofwhy

1.4.20 2:56pm - I highly recommend Primer:, which has a wealth of beautiful examples. Plus, the recent survey by Paul and Elias, geared to economists: #Bookofwhy

1.4.20 6:02am - (Replying to @attilacsordas) Let's discuss by email.

1.4.20 5:54am - (1/ ) What is "counterfactual prediction"? The quantity estimated in: turns out not a Rung-3, but a Rung-2 interventional expression. To avoid confusion & false expectations, I recommend that ML folks adopt the hierarchy in #Bookofwhy. We need to distinguish
1.4.20 5:54am - (2/2) "effects of causes" from "causes of effects" (see and else ML students will come to believe that DL has found a magic way of inferring counterfactuals from data, thus risking an inevitable disappointment.

1.4.20 4:02am - I welcome this new review of #Bookofwhy in American Mathematical Monthly It is written by a hard core statistician unafraid to confess the nature of statistics discomfort with causation.

1.4.20 3:34am - Rabbi Sacks is always insightful and to the point. I would only add that the irrational animosity towards Israel originates from the same swamp: failure to accept a cohesive thriving society not based on class struggle. For elaboration, see

1.4.20 12:12am - (Replying to @attilacsordas) What did you find incomplete in the formal definitions of "sufficient cause" and "necessary cause" as given here: or here: I find them quite satisfactory. Plato.stanford is outdated. #Bookofwhy

1.3.20 6:01am - (Replying to @jonathanborows2) What is "clickbait marketing"? I love the sound of it.

1.3.20 5:58am - (Replying to @mendel_random) I noted, with sadness and humor, that Johnson will have hard time finding qualified applicants for those jobs. (ps. where did you dream up "faithfulness"?) #Bookofwhy

1.3.20 12:50am - I see a much more immediate risk, that @downingstreet will barely find any applicants. I can hardly name a dozen who would qualify, can you? Not because it is hard to explain, but b/c DS folks are too busy doing what they are so good in doing. #Bookofwhy

1.3.20 12:39am - (Replying to @eliasbareinboim) I am trying to interpret the expression "using another DL model". I thought DL prides itself on being "model free". Perhaps you mean "using the estimation prescribed by the CI model" ? #Bookofwhy

1.2.20 11:19pm - (Replying to @UlrichJunker and @sahilsingla47) Thanks

1.2.20 11:18pm - The wrong link, to the "counterfactual prediction" paper was not meant to be an endorsement. I am still not sure the authors are doing counterfactuals in the Rung-3 sense. There is a tendency among OP folks to call interventions counterfactuals; too bad ML folks follow #Bookofwhy

1.2.20 10:31pm - Correcting link to the video: Though it will never correct the mentality of the department that invited this pervert to speak in an "Institute of higher learning". We ARE responsible.

1.2.20 10:21pm - A must watch VIDEO, from San-Diego State University: And what are we doing, as refined academicians, to prevent this poison from igniting our campuses? Have you called out the Chair of that idiotic department? History records both action and apathy.

1.2.20 3:01am - (1/ ) Reflecting back on our DL/CI discussion with @tdietterich, @ylecun, @GaryMarcus etal, we should note that CI folks are invoking the same scientific argument that has brought DL its fame and success. Back propagation and subsequently gradient-based optimization were inspired by
1.2.20 3:01am - (2/ ) machine known in the 1970's. As we go to higher levels of cognition, eg causal inference, it would make sense to ask (and imitate), how the mind preforms this mode of inference, that is, what cognitive
1.2.20 3:01am - (3/ ) templates enable it to manage association, interventions and counterfactuals simultanewously, compactly and swiftly. I know of only one such template: Structural Causal Model. Thus, instead of hoping that CI templates would pop up spontaneously from DL, let us ask first
1.2.20 3:01am - (4/4) what should they look like? How should we represent them, and what we would be able to do once we get them. #Bookofwhy

1.2.20 12:36am - Glad to see that subtitles were added to my podcast with Lex Fridman I can now understand my Hebrew accent. #Bookofwhy

1.1.20 3:53pm - (Replying to @AdanZBecerra1 @Lester_Domes and @nature) I was called "disruptive" in 1st grade of school, sent home, and was back only three weeks later, after seeing an oppressive psychiatrist who vowed that I was ready. Evidently, I wasn't. #Bookofwhy

1.1.20 2:49am - (Replying to @maximananyev) How many Masters does it take to correct what some Metrics refuse to do? #Bookofwhy

12.31.19 11:29pm - (Replying to @BariFaisal) I assume you refer to #Bookofwhy? My thoughts on empirical research are summarized here:, and on RCT, here: Warning: Graph-shunning economists are depriving their students of ever combining experimental and observational studies.

12.31.19 7:49pm - (Replying to @Isalomaki) Yes, "Be" means "We", not "Me, Me, Me".

12.31.19 4:56am - (Replying to @HenMazzig)` Isn't it what congress-woman Tlaib has been howling all along?

12.31.19 4:45am - (1/ ) My New-Year greeting this year is again in a form of a poem/song, through which I wish all readers:
"Make it be, Make it be, Pleading: "Make it be", All for which we pray Make it be."
In 1973, Nomi Shemer's, set out to translate the Beatle's song "Let it be" into Hebrew. But
12.31.19 4:45am - (2/ ) then the Yom Kippur war broke out, and what came out her pen was a prayer that swept the country with immense yearnings for peace. It still resonates today in every street and on every occasion:
The summer ends, the journey's over, Let them please at last come home...
12.31.19 4:45am - (3/3) All for which we pray: "Make it be".
You can hear the entire song about 7:10 minutes into this video.
Happy New Year! and "Make it be!"

12.30.19 1:55pm - (Replying to @AwokeKnowing @ylecun and @LeCun) Sure. The math of woodwork is not too complex, so I dont see why it cannot be expressed as output of the math of metallurgy, given the unique metallurgical properties of wood. #Bookofwhy

12.30.19 1:39pm - (Replying to @vatsal_maru @ylecun and @lexfridman) Solving ONE toy problem in causal inference is worth 1000 conversations, interviews, debates & what-have-you about what others do. Take, say, Simpson's paradox and analyze it through the math of "gradient-based optimization", but stick to the toy! #Bookofwhy.

12.30.19 1:06pm - (Replying to @ylecun) Please listen carefully to what the carpenter is saying:
* Carpenter: Sure, you can do the nails, I'll do the woodwork, together we can build a house.
He should also add that the math of metallurgy is not very helpful in carpentry, a new math is needed. #bookofwhy

12.30.19 6:38am - (Replying to @ElliotMalin) Wishing you and your lucky bride many years of happiness. And may your commitment to Israel be a constant inspiration to you and to your children.

12.30.19 4:48am - (Replying to @PolandCherieM and @oralassila) In #Bookofwhy I was just summarizing half a century of zero-progress in external-validity research, stuck for lack of language. DL, in contrast, has shown immense progress, but lack of language is still a fundamental impediment, and is beginning to show its face.

12.30.19 4:06am - A strange thought on what DL can and cannot do.
` If "X can do Y" whenever we need X to do Y, then transistor experts "can do" AI, and nail-makers "can do" furniture.

12.30.19 3:33am - (Replying to @ArnaudMegret) And naive me assumed that every ML researcher read #Bookofwhy, with just a few hard cases who remained unconvinced. No wonder @ylecun 's followers keep asking: "Can't we just "extend" DL a bit to cover causality?" I don't believe any reader of #Bookofwhy would ask such a question.

12.30.19 3:13am - (Replying to @ArnaudMegret and @LeCun) This paper summarizes our latest findings on how much information one can get out of missing data, both MAR and non-MAR As you point out, the question is causal, and it is hard to imagine how any statistical analysis can obtain these results. #Bookofwhy

12.30.19 2:05am - (Replying to @AngeloDalli @ylecun and @LeCun)
* Metallurgist: My nails just did a table, they can do a house?"
* Carpenter: Sure, you can do the nails, I'll do the woodwork, together we can build a house.
* Metallurgist: No! The nails should do it. They can perhaps be extended to handle the woodwork.

12.29.19 4:13pm - (Replying to @ylecun and @LeCun) If we have interventions, we do not need CI, we just fit the observed data (eg by DL) and we're done. CI is needed to answer questions that cannot be answered directly from data, eg "Find the effect of X, when I only have interventions on Z". A new math is needed here. #Bookofwhy

12.29.19 6:48am - (Replying to @ngutten @jsusskin and @LeCun) Sure! Everyone (except perhaps Yann) understands that the goal of DL is to fit a function to 10K+vars, and that CI has a totally different goal (requiring totally different math): to answer causal questions about reality ASSUMING that DL folks succeed in their fitting efforts.

12.29.19 6:36am - Today's madness could not be more vividly portrayed than through this record of @KenRoth , the Orwellian watchman of (the world's) Human Rights.

12.29.19 6:27am - (Replying to @SussexFriends) Is this moral pervert still talking? After all that he did to embolden hate against Israel? May my last words to him be: Nes Gaddol Haya Po (in UK).

12.29.19 5:33am - (1/ ) (Replying to @ngutten @jsusskin and @LeCun) We should discourage researchers from thinking that 'to do CI with regression requires an extra step'. This would lead them into believing that doing CI with regression is not an oxymoron, that the extra step is trivial and would pop up spontaneously if only they study more
12.29.19 5:48am - (2/ ) (Replying to @yudapearl @ngutten and 2 others) and more regression. It won't. On the contrary. Today, the best experts on regression are the least likely to understand what is needed to do CI. I foresee the same fate awaiting students of DL. My advice: start afresh, with CI, and resort to DL only if needed. #Bookofwhy

12.29.19 3:59am - I've hoped to celebrate the 8th night of Chanukka on a joyful note. In the wake of the Monsey stabbing, I must retreat to what I wrote in 2009 (in the WSJ): The normalization of evil begins with the normalization of hate, and our leaders are not watchful:

12.29.19 2:32am - (Replying to @Imamofpeace) That's essentially what Linda Sarsour told Bernie Sander. He believed her, assuming she represents the next wave of votes. The consequences are in the streets of NYC.

12.28.19 5:38pm - (Replying to @KordingLab @GaryMarcus and 7 others) As you can see here:, I love everything about the economists tradition, except their latest betrayal of that tradition, and their cultish avoidance of tools that operationalize that tradition. See eg #Bookofwhy

12.28.19 5:14pm - (Replying to @KordingLab @GaryMarcus and 7 others) I am sincerely trying to understand what it IS, or what it means to YOU, and you are slapping me with "denying the validity". Unfair! #Bookofwhy

12.28.19 5:08pm - (Replying to @KordingLab @GaryMarcus and 7 others) For the sake of amazing clarity I have been trying to avoid the term "different flavor", because it has been misused to create the impression that there are differences in substance where there are none. @Bookofwhy

12.28.19 4:06pm - (Replying to @KordingLab @GaryMarcus and 7 others) The arguments are perfectly solid. I am just not clear what econ. practitioners call "OV bias equation", and how they use the equation to reach new heights. #Bookofwhy

12.28.19 3:54pm - (Replying to @edwardsjk @EpidByDesign and 11 others) I take it that the idea is to "combine" observational and experimental studies, rather than exclude one in favor of the other. Are the results of "combining" different than those of "fusing"? As in eg ?#Bookofwhy

12.28.19 3:39pm - (Replying to @KordingLab @GaryMarcus and 7 others) A sincere question: What is the "omitted variable bias equation"? How does one realistically "apply it to real world reasoning?" I honestly havn't seen it applied, but I might have missed the equation title. #Bookofwhy

12.28.19 3:24pm - (Replying to @KordingLab @GaryMarcus and 7 others) I would love to join you in admiration of "econ style CI" if I only knew what it is, what it is "complementary" to, and how it "beautifully complements" pieces that are missing elsewhere. Curious. #Bookofwhy

12.28.19 6:14am - (Replying to @oralassila) It is not an implicit prediction as much as it is an explicit concern. If we continue to invest most of our resources in the science of nuts and bolts, engrossed and intoxicated by its successes, our ability to build engines will be delayed. #Bookofwhy

12.28.19 4:16am - (Replying to @PolandCherieM) Thanks for posting this illuminating paper on the history of regression. Biased by the lens of causation we emphasized the odd fact that Galton sought a causal explanation to Darwin's theory and ended up abandoning causation to the mercy of correlation. What a miss! #Bookofwhy

12.28.19 3:55am - In addition to introducing new methods of generalization, this paper also identifies and solves a new and important problem of learning causal models from both observations and interventions. Highly recommended. #Bookofwhy

12.28.19 3:14am - To all who celebrate the 6th night of Hanukkah with my friends and colleagues in Israel, I am sharing a song that we used to sing since Kindergarten:
A miracle did not happen to us, We have not found a vessel of oil, We carved the rock till we bled, And there was LIGHT!

12.28.19 3:03am - Variation on a painful, left-leaning theme: The only thing less politically convenient for leftists to recognize than left-wing antisemitism is left-wing Zionophobia and Palestinian elimination-ism.

12.28.19 2:47am - I am retweeting this reply, for it crystallizes my position in the latest conversation on the relationships between DL (deep learning) and CI (causal inference) with @tdietterich , @ylecun , @GaryMarcus , @rodneyabrooks and significant others. #Bookofwhy

12.28.19 2:06am - (Replying to @moultano @bradpwyble and @GaryMarcus) It is extremely difficult to make precision nuts and bolts, and it is by no means "trivial". But the science of making nuts and bolts is not the science of making engines, and a community engrossed in the former will be limping on the latter. #Bookofwhy

12.27.19 8:08am - (Replying to @kudkudakpl @GaryMarcus and @moultano) A different approach is needed. Now it is my turn to ask "out of curiosity": Why do you keep saying "doing DL"? Is it because you enjoy approximating functions? Or because you want to ride the DL hype? Seriously, I am truly curious. #Bookofwhy

12.27.19 8:01am - (Replying to @databoydg and @tdietterich) Same can be said about linear algebra. Current "approaches" are insufficient but, eventually, everything AI will achieve in the future will use linear algebra to some extent, So, linear algebra will spontaneously spawn intelligence. Tough? Don't bet against DL. #Bookofwhy

12.27.19 7:39am - (Replying to @kudkudakpl @GaryMarcus and @moultano) Good question. What I mean by "mathematical impossibility" is w/o causal (extra-data) assumptions of some kind. The paper u cite makes such causal assumptions and the study of what assumptions are needed has been the central focus of CI research in the past 3 decades. #Bookofwhy

12.27.19 7:29am - (Replying to @databoydg and @tdietterich) Why assume anyone pretends they can't exist together? I just wrote that I have always assumed DL is almighty and can perfectly approximate any function and, so, it can exist together with CI, approximating galore all functions that CI says should be approximated. #Bookofwhy

12.27.19 7:05am - (Replying to @kishkushkay)` When was that, Kay?

12.27.19 6:55am - (Replying to @memosisland @GaryMarcus and 2 others) @tdietterich wrote that DL's aim was to develop "science and engineering of intelligent systems." I removed "engineering" from his quote, because I want to focus on the "science", where CI has made some modest contributions. #Bookofwhy

12.27.19 6:41am - (Replying to @moultano and @GaryMarcus) My expectations about DL never ended up wrong, because I have always assumed DL is almighty, and can approximate perfectly every function it is given, regardless how complex. But for DL to spontaneously produce CI, it's not implausible, its a mathematical impossibility #Bookofwhy

12.27.19 5:26am - (1/ ) There is more to @GaryMarcus argument than taking credit. If the goal of DL was indeed as ambitious as you describe it ("science of intelligent systems") than it failed miserably by not realizing the barriers to rising above Rung-1 (ie., function fitting). If, however, the
12.27.19 5:26am - (2/ ) goal was more modest, say to perfect the art of function fitting, then there is no room for claims that CI will eventually become part of the DL umbrella. The distinction between "CI will be under the DL umbrella" vs. "DL is part of the CI umbrella" is not about credit. It's
12.27.19 5:26am - (3/ ) about the agenda of AI research in the next decade. Saying "CI is merely an extension of DL methodology" amounts to telling educators, funders and the public: "Let us continue doing what we have been doing all along, CI will emerge organically." It won't! And the result:
12.27.19 5:26am - (4/ ) AI progress will be slowed, and the disappointment will be painful. In contrast, saying: "DL is a component of CI" amounts to telling funders and educators: "We must educate a new breed of DL researchers, capable of tackling new challenges. Things as usual just won't work!"
12.27.19 5:26am - (5/5) I believe the second alternative is more constructive. "Things as usual" leads to "Things as usual" unless there is a paradigm change & a community awakening to the limitations of "Things as usual". Luckily, we can formalize those limitations. Let's overcome them! #Bookofwhy

12.27.19 3:46am - Happy new year! To all readers who are hoping for one, me included. Glad Israel remains the world's beacon of moral clarity, something the world has forgotten exists, and finds it hard to stomach. Happy new year!!

12.27.19 3:15am - Thanks for posting this interesting paper. I've never imagined that causal graphs will find applications in such esoteric fields as Visual Dialogue. But I am known to suffer from occasional lapse of imagination. #Bookofwhy

12.27.19 2:38am - (1/ ) (Replying to @tdietterich) Disparity in levels is one source of the confusion. You congratulate DL engineers for their achievement in approximating such complex functions, and I say: it is ONLY "curve fitting". We are both right! In the past 3 decades I took function approximation to be a done deal.
12.27.19 2:45am - (2/ ) (Replying to @yudapearl and @tdietterich) How? Every time my math ended up with a probability expression, say E(Y|x,z), I labeled it "solved", namely, DL folks will find a way of approximating this function from data. Done deal! Let's look at the more challenging task of reducing our causal questions to probabilities!
12.27.19 2:56am - (3/ ) (Replying to @yudapearl and @tdietterich) Today, that we know almost everything about what can or cannot be reduced (meaning what DL can or cannot do) we find DL folks awakening to, and dismissing the main challenge with: "We can fit everything!" It's true, they only need someone to tell them what to fit.#Bookofwhy

12.27.19 2:19am - (Replying to @eyad_nawar) If we have 1&3 you do not need controlled experiment. If we don't, we can't move beyond "curve fitting". This is something DL folks are beginning to realize, in their own unique pace. #Bookofwhy

12.27.19 2:05am - (Replying to @steventberry @PHuenermund and @ChrisAdamsEcon) The confusion about "what to control for" is symptomatic of a fundamental methodological neglect econometric is currently recovering from. I compiled a few more symptoms in: For survey and remedy, see: #Bookofwhy, @ben_golub

12.26.19 6:54am - Arnold Roth is both a friend and a grieving comrade. His daughter Malki and my son Daniel were victims of the same wave of hate that swept our planet in 2001-2002. Please listen to his plea. Judea Pearl

12.26.19 6:17am - (Replying to @guillefix @maier_ak and 5 others) What is not available to DL training is not "data" but "information" on how to get the appropriate data. For example, that "age" is a confounder, hence, estimate the age-specific effects, not aggregate effects. See Simpson's paradox, #Bookofwhy p.211, or

12.26.19 6:05am - (Replying to @bioinfochat @tdietterich and 3 others) Agree, "regression is here to stay" does not imply "DL is here to stay", I should have said "Some form of function approximation is here to stay", which is tautological and elevates DL to a special status it rightly deserves. #Bookofwhy

12.26.19 5:44am - The Middle East Studies Association (MESA) sent a letter to President Trump in which they decide how Jews should define themselves. The verdict: Jews are not a people of common national origin. Their logic betrays their wisdom --a true academic gem.
12.26.19 5:44am - (2/ ) The logic goes: Since not ALL Jews share a common national origin, therefore, NO Jew, including the vast majority who DO, should be treated as such. May the 5th candle of Hanuka remind those academic "experts" how Jews have been defining themselves for the past 2200 years.

12.26.19 3:06am - (Replying to @LuisMateusRocha and @LeCun) Thanks, but this is over my head. Think about a naive AI fellow who wants to implement one of those "non-inductive" machine. Where should she start?

12.26.19 2:55am - Who said John Searle will never pass the Chinese Room Turing Test? See #Bookofwhy page 38-39.

12.26.19 2:35am - (Replying to @LuisMateusRocha and @LeCun) I'm not familiar with the technical details of this literature. Can you Tweet what information "anticipatory system" has that #induction machines do not, where it gets it, and how it represents it. Thanks #Bookofwhy.

12.26.19 2:02am - (1/ ) (Replying to @tdietterich @Plinz and @GaryMarcus) The point is that the term "functional approximation" is more informative to the engineer than an account of flexibility. It tell the engineer to stop asking how good the approximation and start asking whether she approximated the right function, and how to get the correct
12.26.19 2:12am - (2/ ) (Replying to @yudapearl @tdietterich and 2 others) input-output pairs. It also means that @ylecun statement "DL is here to stay" is tautological, akin to "programming is here to stay". What counts is whether we write the right program. And when we do, we don't label it "Turing Machine" #Bookofwhy

12.25.19 4:39pm - (Replying to @ylecun @BethCarey12 and 2 others) Just for the music: Replace DL with "regression", and see how things still stick, and the music is still flowing soundly and harmoniously, even more so, for we know so much about "regression" and its limitations. BUT: Is "regression" is the right scientific model? NO! #Bookofwhy

12.25.19 2:30pm - (Replying to @tdietterich @Plinz and @GaryMarcus) You'r speaking from a function-builder perspective. But from the end-user, input-output, what have we got? A function that approximates the actual relationship between input and output. It need not capture the pain, labor, flexibility and sophistication of the builder. #Bookofwhy

12.25.19 5:19am - (Replying to @maier_ak @hardmaru and 4 others) When we talk "algorithms" everything seems to be doable in DL or NN etc. But when we talk about "information" the limitations become clear -- information cannot be created by processing data, it is either there or not. As Cartwright said: "No causes in, no causes out." #Bookofwhy

12.25.19 3:25am - (Replying to @recursus @basorot and @LeCun) Fine, but "working out the entailments of a model" requires a data structure to represent that model. Once you commit to such representation, Bingo! You are doing CI! And if you still want to keep your memebership in the DL club, fine, we wont excommunicate you. #Bookofwhy

12.25.19 2:15am - (Replying to @recursus @basorot and @LeCun) Sorry to disappoint, but the "model" cannot unfortunately "be found through a generate-and-test process", b/c there is nothing in the training data that can refute a bad model. It can only be tested when we make a bad policy decision and, then, it's too late. #Bookofwhy

12.25.19 1:24am - To all readers who celebrate it, Merry Christmas from me and from my family in Jerusalem. I fell in love with Christmas the moment we arrived to US, 1960, primarily b/c it reminded me so much of Hanuka as it was celebrated in my home town, with songs and lights from every window.

12.25.19 12:56am - (Replying to @hardmaru @ngutten and 3 others) These proposals reinforce my conclusion that some DL folks have not internalized the Ladder of Causation and its implications. One of these says: No matter how you squeeze the data, no matter how smart the squeezing, you can't get causal information out of it. Sorry #Bookofwhy

12.25.19 12:11am - (Replying to @ngutten and @jsusskin) Interesting viewpoint. Trouble is, regression is such a small part of CI that to say "To do CI with regression requires an extra step" is almost like saying: "To do CI with algebra requires an extra step". Its better to say "Do CI first, add NN if needed" @ylecun #Bookofwhy

12.25.19 12:11am - (Replying to @ngutten and @jsusskin) Interesting viewpoint. Trouble is, regression is such a small part of CI that to say "To do CI with regression requires an extra step" is almost like saying: "To do CI with algebra requires an extra step". Its better to say "Do CI first, add NN if needed" @ylecun #Bookofwhy

12.24.19 11:10pm - (Replying to @ylecun and @LeCun) And there is another reason why I want to convince YOU, rather than Leon. I have noticed a day-and-night difference between folks who actually solved a toy problem in CI and those who know all about how others do it. If you are among the former, Leon would do. #Bookofwhy

12.24.19 10:57pm - (Replying to @ylecun and @LeCun) And there is another reason why I want to convince YOU, rather than Leon. I have noticed a day-and-night difference between folks who actually solved a toy problem in CI and those who know all about how others do it. If you are among the former, Leon would do. #Bookofwhy

12.24.19 10:57pm - (Replying to @Plinz and @GaryMarcus) And what will be lost if we drop the "compositional" and call it just "function approximation"?? or, God forbid "curve fitting"? (see footnote b in #Bookofwhy

12.24.19 10:47pm - (Replying to @GaryMarcus) Great questions, @BaryMarcus! May I add a 5th question: What will be lost to AI if we replace the term DL with "super-efficient way of doing regression" or simply "regression"? A lot will be gained, b/c we know so much about what can and cannot be done w/ regression. #Bookofwhy

12.24.19 10:22pm - (1/ ) (Replying to @ngutten @jsusskin and @LeCun) Sure. A NN trained on specially prepared data would capture the features that the preparation meant it to capture. How much preparation can RL do? See Tweet: By deploying interventions in training, RL allows us to infer consequences of those interventions, but ONLY those
12.24.19 10:22pm - (2/ ) (Replying to @yudapearl @ngutten and 2 others) interventions. A causal model is needed to go BEYOND, ie, to predict consequences of actions not used in training, or even combinations of actions used in training. Are NN useful in CI? Sure! Same as asking "is regression useful in CI?" Sure!. But is NN different? #Bookofwhy

12.24.19 9:12pm - (Replying to @basorot and @LeCun) Wait, Wait! This is an interesting point!. You propose to call the model "part of processing". Fine. Have you seen a DL system given a model and, after processing it with the data, answer questions it could not answer before? This is a critical ingredient ML folks lack #Bookofwhy

12.24.19 9:03pm - (Replying to @jsusskin and @LeCun) If we go beyond the definition, we find Yann saying: Don't say "DL doesn't do causal inference" when you really mean "a plain, supervised neural net does not spontaneously discover causal relationships." The correct answer is: "DL just doesn't! Regardless of the NN!" @Bookofwhy

12.24.19 8:51pm - (Replying to @jsusskin and @LeCun) If we go beyond the definition, we find Yann saying: Don't say "DL doesn't do causal inference" when you really mean "a plain, supervised neural net does not spontaneously discover causal relationships." The correct answer is: "DL just doesn't! Regardless of the NN!" @Bookofwhy

12.24.19 8:23pm - (Replying to @basorot and @LeCun) If you cannot get answers for questions by processing the data, and those questions ought to be answered by a sensible agent, it means that the sensible agent uses some information that is not in the data itself. This extra-information is called "model", not "data."#Bookofwhy

12.24.19 7:01pm - This bold defense of deep learning says that some DL folks have yet to internalize the Ladder of Causation, eg Can any DL-insider think of a good way to convince @Lecun that some questions cannot be answered from data alone, no matter what? #Bookofwhy.

12.24.19 5:12am - (Replying to @HenMazzig) Well said. However, I was informed that Arafat started the erasure of the Jewish history already in 2000, at Camp David. See . Also, the Palestinian claims are weakest when "indigeneity" is defined as an intellectual-cultural state of mind.

12.24.19 4:43am - (Replying to @jrgptrs and @JustinSandefur) In 1987 David Freedman wrote "As others see us", which had a soul-searching impact on social scientists. Econometrics is waiting for an insider to write such a paper. I volunteer to play one of the "others", but the field is begging for a courageous insider. #Bookofwhy

12.24.19 2:27am - (1/ ) (Replying to @jrgptrs and @JustinSandefur) 1/ I perfectly understand the culture that reinforces your arguments, but I beg to question its logic. The fact that "no referee requires us to use [the] Data Fusion approach." does not negate the possibility that the methods used by that culture are grossly outdated. In fact,
12.24.19 2:37am - (2/ ) (Replying to @yudapearl @jrgptrs and @JustinSandefur) readers on this Twitter post have been decrying the insular, echo-chambered character of economics literature in general, and in the area of external validity in particular. So, I am glad you are looking into this issue from a non-cultish perspective. And, BTW, "Data Fusion"
12.24.19 2:42am - (3/ ) (Replying to @yudapearl @jrgptrs and @JustinSandefur) is not "my approach" nor is it something you may choose to apply or ignore. The findings of "Data Fusion" analysis are universal, regardless of how you choose to extrapolate across environments. The environments either permit extrapolations or not and, ignoring the latter
12.24.19 2:49am - (4/ ) (Replying to @yudapearl @jrgptrs and @JustinSandefur) may lead to errors regardless of what referees say or not say. I trust that, after looking into the state of art, you will join your colleagues' efforts to elevate econometrics to the age of modernity. #Bookofwhy

12.24.19 12:14am - (1/ ) This paper introduces an interesting refinement of selection diagrams: If one is interested in transporting a single contrast (say Risk Difference) and is willing to make parametric assumptions to match this aim, then selection diagrams can be pruned
12.24.19 12:14am - (2/ ) so as to reduce the number of covariates involved. This refinement is similar in spirit to that proposed in, which leverages knowledge of how mechanisms interact to produce the outcome. #Bookofwhy
12.24.19 12:49am - (2/ ) (Replying to @yudapearl and @DavidHarrisAJC) This means that, seeing any such claim opens up an opportunity to expose, with facts, figures and dates, the glaring recentness of the claimant's historical heritage.

12.23.19 10:38pm - (Replying to @DavidHarrisAJC) And let's not forget that claims of "no Jewish historical tie w/ Israel & Jerusalem!" are fabricated for a reason; to justify an embarrassing 'no historical tie' on the side of the claimants. As described here

12.23.19 7:31pm - We, who are bound to earth, salute you @Astro_Jessica for showing us how cosmic imagination can enrich a humble tradition, of a tiny people, who did not have much to offer the world, except learning, memory and imagination.

12.23.19 7:16pm - The legitimization of proxies is treated here and here We call it "Measurement Bias and Effect Restoration". BTW, the conditions established are not unique to Pearl-stale analysis; they govern ALL analyses, no escape. #Bookofwhy

12.23.19 4:26pm - (Replying to @AngeloDalli @GaryMarcus and 2 others) I'll second the conclusions. Now, let's do it. That's what the do-operator calls on us to do. #Bookofwhy

12.23.19 3:22pm - (Replying to @EpiEllie and @smueller) If I were a mentor to 5-yr old, I would say: Isn't it amazing? With the same tools we used to determine backdoor conditions you can now determine how to re-weight data to estimate target effects? This possibility will not occur to an economist in the next 17 years! #Bookofwhy

12.23.19 3:09pm - (Replying to @emzanotti) She doesn't. Hanuka represents Jewish sovereignty in some part of the middle east, an idea that @IlhanMN Omar has vowed to eradicate, in blunt betrayal of her Minnesota voters.

12.23.19 6:31am - (Replying to @stuartbuck1 @Jabaluck and @metrics52) Inverting matrices is a good analogy. I would use "solving equations algebraically" vs. trying out all possible solutions and see which satisfies the equations. We can do it by hand, true, but where would science be today w/o algebra? #Bookofwhy

12.23.19 5:10am - (Replying to @charleendadams) Ocho Candelikas is my favorite Chanukah song. Inviting all our Spanish speaking readers to join us in lighting the candles and celebrating the defeat of darkness. Here:

12.23.19 2:13am - Readers elevated by the Holidays spirit are invited to a sing-along in Hebrew, ending with a Chanukah song we used to sing in kindergarten: Sevivon Sov Sov Sov, about 11 minutes into this video Hallelujah!

12.22.19 5:16am - (Replying to @totteh @jttiehen and @keithfrankish) I wrote "sifting", not "shifting", because I am always hoping for some reader to show me a nugget that I have neglected in my combing of this literature. #Bookofwhy

12.21.19 5:52pm - For readers baked in traditional philosophical theories of causation, who wish to see how these gel with modern formalisms, this thesis provides a panoramic view: I am still sifting this tradition for a gold nugget worth adding to SCM #Bookofwhy

12.21.19 12:49pm - (Replying to @mattshomepage and @Elias) More than that! The completeness of the do-calculus is nice, but it does not provide us a procedure for finding an identifying pattern. The algorithm (summarized here: actually finds it, whenever such a pattern exists. #Bookofwhy

12.21.19 11:05am - (Replying to @mattshomepage) The answer is YES. @Elias algorithm can be proven to be "complete" namely, guaranteed to discover ALL doors, where by "doors" we mean patterns that allow identification. #Bookofwhy

12.21.19 10:38am - (Replying to @mattshomepage) I do not buy the "very specific structure" limitation. The class of structures satisfying the front-door condition is not smaller than those satisfying backdoor condition. It is only that the latter is in more common use. #Bookofwhy

12.21.19 10:33am - (Replying to @mattshomepage) Well said, and it entices me to add: When you wish to express WHAT you know, you better express it the WAY you know it, that is, the way it is stored in your mind, not the way it would justify favorite statistical routines. #Bookofwhy

12.21.19 4:41am - (Replying to @PWGTennant and @medrxivpreprint) I am anxious to see the first paragraph, where DAGs are introduced. eg. DAGs are .....

12.21.19 4:20am - (Replying to @ang_hermann) Good luck @ang_hermann on your French translation project. I know it is a work of love, else you would not have undertaken it. I'm looking forward to see @Bookofwhy on French book-shelves, and hear France communicate with Portugal in DAGs. PS. a Spanish one is in the making too.

12.21.19 3:04am - (1/ ) (Replying to @ErakatSaeb) This is not a joke! It is in fact the cause, the root, and the essence of the Palestinian tragedy. Unable to celebrate any holiday connected to the land to which they claim sole ownership. Unable to chant a single hymn authored in the days of Jesus or Judas Maccabeus,
12.21.19 3:09am - (2/ ) (Replying to @yudapearl and @ErakatSaeb) lacking any cultural connection to those days, Palestinians been laboring to fabricate such connection by molesting the heritage of their neighbors, and hoping to be taken seriously and well-intentioned if/when they decide to come to the negotiating table again. If only ...
12.21.19 3:14am - (3/3) (Replying to @yudapearl and @ErakatSaeb) If only @ErakatSaeb understood what this circus acrobatic does to Palestinian posture.

12.21.19 2:04am - (1/ ) Halleluya, It's Hanuka! Sunday night we will be lighting the first candle to commemorate the Maccabees revolt, Jerusalem, 161 BC. Join me in a holiday that inspires all who cherish the ideas of freedom and self-determination. My grandson asked me why we make such a big fuss l
12.21.19 2:04am - (2/ ) about this holiday. I told him Hanuka is our TRUST DEED to the birthplace of our history, more solid even than the ancient synagogues they excavate in Israel, or the arch of Titus in Rome. Why? Because stones can be faked, not so a continuous celebration for 110 generations.
12.21.19 2:04am - (3/3) I wrote about this aspect of Hanuka when the LA Times asked me "what does Israel mean to you?" and why understanding Hanuka is so essential for dreaming any peace prospect between these two equally indigenous peoples.

12.21.19 12:36am - Halleluya!! Good tidings! Kindle has finally uploaded a revised edition of PRIMER with all the errata smoothed to perfection. Enjoy. As to the hard copy, publishers try to keep this information very very secret. But I'll Tweet when it happens #Bookofwhy

12.21.19 12:14am - (1/ ) Gratified to see researchers committed to decision making adopt modern tools of causal inference. I can tell right away that the authors have not graduated from UCLA. How? (1) Consistency is an "assumption" in PO, not so in SCM, where it is a "theorem",
12.21.19 12:14am - (2/2) and (2) Graphs/potential-outcomes symbiosis is not sacrilegious in SCM; it is in fact the norm; the former to express what we know, the latter what we wish to know. See #Bookofwhy (though it may be sacrilegious in some islands of PO, eg Lilliput and secret others)

12.20.19 11:41pm - Yes, worth keeping in mind, and add to it: Hypothesizing means unabashedly committing to a representation (e.g., a model) from which we can deduce how things operate, when the need arises. #Bookofwhy

12.20.19 3:33am - (Replying to @maximananyev @Jabaluck and @metrics52) Do you think this great paper gives potential users any idea about how to select variables for valid reweighing? (even if we knew exactly how study-participation was determined in MA and in NJ). #Bookofwhy

12.20.19 1:28am - In the island of Lilliput they don't teach addition. Why? Because the problems they truly truly wish to solve involve multiplication; addition alone cannot handle. That's in the legendary island of Lilliput. #Bookofwhy

12.19.19 8:25pm - (Replying to @Jabaluck and @metrics52) Ask not what I did wrong nor what I did right, ask what YOU can do to elevate your field to new heights. #Bookofwhy

12.19.19 7:36am - (1/ ) True, Journals normally do not bid and do not solicit. However, what makes an enlightened editor ENLIGHTENED is his/her ability to spot promising new develpments and invite authors to write a review article for readers who have not been exposed to it. Phil Dawid, for example
12.19.19 7:36am - (2/ ) invited me in 1995 to write an article to Biometrika, because he felt that it was about time that Stat readers will learn something about graphical models. He probably faced stiff resistance by reviewers and the established elite. He did it, and his leadership made an impact.
12.19.19 7:36am - (3/ ) I wrote at least a dozen papers thus invited by enlightened editors. You don't have to be a genius to see that your field lags behind in a given area, you need only be a leader to do something about it. This is what I meant by "bidding." Perhaps econ. editors are still
12.19.19 7:36am - (4/ ) fighting for the honor to be the first? Or perhaps they have not been informed of the paper. Alert them. Some editors need a jolt. Play on their patriotic duties to their readership. Econ. students can't afford another decade of arrow-phobic prohibition. Can they? #Bookofwhy

12.19.19 3:49am - (Replying to @SadiqKhan) Commending you on your courage and good will. But to be effective, please heed to an advise of someone who has studied the anatomy of anti-Semitism. Change your statement to read: "a hostile environment for anti-Semites, Islamophobes and Zionophobes. @EinatWilf

12.19.19 2:42am - I bet you already received 4-5 invitations from top Econ journals. Seriously, one can argue till dawn (as @jabaluck & @metrics52 ) that economists do not need new identification strategies. But I dont know one who'd argue they don't need external validity or data fusion.#Bookofwhy

12.19.19 2:32am - (Replying to @Meetasengupta and @prem_k) Would love to discuss those possibilities with you, but my family will be descending on us on Chrismass, so I can't make any plans in advance. Let's play it by ear.

12.19.19 12:12am - (Replying to @Meetasengupta @prem_k and 3 others) I believe we can go a step beyond "try to narrow the gap", since we are in the age of causation, where "trying" is just not good enough. We can at least formalize when gaps are narrowable, what information is needed to narrow them, etc. We have the language to do it! #Bookofwhy

12.18.19 11:40pm - (Replying to @prem_k @Meetasengupta and 3 others) I am not sure what "Evidence Based" people count as "evidence". I know that N. Cartwright, one of the champions of EB has also complained bitterly about external (non)validity of RCT's, and I've tried to introduce her (and Deaton) to S-diagrams: #Bookofwhy

12.18.19 11:28pm - Evidently, wines do taste better with age. As I re-read the 1995 paper on instrumental inequalities,, I can't help but wish that IV's would be introduced today with the same clarity and freshness as they were treated then, in 1995. #Bookofwhy

12.18.19 10:47pm - To readers who asked whether I heard back from Miguel why he goes back to PO, given that it is so easy to generalize in DAGs,, the answer is not yet. I think Harvard folks love DAGs, but still have problems swallowing do-calculus. Who knows? #Bookofwhy

12.18.19 5:20pm - (1/2) Glad to see this ancient inequality finding useful applications in Mendelian Randomization. Truthfully, I knew already in 1995 that it would be resurrected some day, but I wrote it for economists, who loved IV and bemoaned: "it can't be tested". I thought they would jump
12.18.19 5:20pm - (2/2) from joy and celebrate the testability of their beloved IV. Little did I know that economists have been waiting for epidemiologists to take the lead. #Bookofwhy

12.18.19 3:01pm - This paper may just be what econometrics has been waiting for in its effort to catch up with the causal generation. I envision dozens of editors bidding on the right to publish it in their journals. Imagine what such paper could do to the reputation of Econometrika. #Bookofwhy

12.18.19 1:24am - (1/ ) Our success of showing readers how re-weighing works in moving across populations entices me to show how it works in correcting sampling-selection bias. Assume we preferentially select subjects for a study based of an arbitrary set of characteristics (some measured & some not) 12.18.19 1:24am - (2/ ) , and we wish to estimate P(y|x) in the target (unsampled) population. Can the DAG tell us when we can do it? Yes. Whenever we can measure a set Z of variables such that: (1) {X,Z} separates S from Y and (2) The conditional distribution P(z|x) can be estimated for the target 12.18.19 1:24am - (3/ ) (unsampled) population. If these hold, then the re-weighing formula P(y|x) = SEM_z P(y|x,z, S=1) P(z|x) gives us an unbiased estimate of P(y|x) in the target population. Illuminating examples, generalizations and refinements are given in 12.18.19 1:24am - (4/4) Note that the only difference between this re-weighing formula and the one used in transportability is the weighing term P(z|x) vs. P(z). Note also that even @EpiEllie 5-yr old can enjoy and appreciate the power of selection diagrams; how else would one select Z?? #Bookofwhy

12.17.19 6:36pm - Our success of showing readers how re-weighing works in moving across populations entices me to show how it works in correcting sampling-selection bias. Assume we preferentially select subjects for a study based of an arbitrary set of characteristics (some measured & some not)

12.17.19 5:59pm - (1/ ) Great to see Uber joining the causal era. There is one glitch in this article which might be misleading to the novice - the word "approach" appears 19 times. Why misleading? Because "approach" connotes an "option" under the analyst control. Yet the 19 methods discussed
12.17.19 5:59pm - (2/ ) are not under our control but are "opportunities" made available to us by (what we believe is true about) reality, ie, our model of reality (DAG). Once we have a model of reality we cannot speak about the 19 methods as "approaches," ie, as if they were our options. #Bookofwhy

12.17.19 6:31am - (Replying to @jeremyphoward and @calabi_and_yau) Students of causality would read this paper with a constant question in mind: How do you climb the Ladder of Causation, from predictions, to effect of actions, to counterfactuals (eg would this customer be swayed by this discount). I guess it is all in the model used. #bookofwhy

12.17.19 6:25am - (Replying to @HenningStrandin) Thanks for posting this interesting summary. Just in case you find it relevant, I have found comfort in escaping the circularity trap through another route. It is in Section 33.5.3 of this paper: #Bookofwhy

12.17.19 12:54am - (1/3) I am retweeting Charlene's insightful and personal testimony from the holy city of Bristol. The city where anti-semitism does not exist, and whatever does exist has nothing to do with Corbyn, and whatever has to do with him is Jewish invention or Israel's fault.
12.17.19 12:54am - (2/3) Bristol is a city of highly devoted Corbynites who helped turned a true "Labour Party" into a dangerous and un-electable cult of directionless excuse makers. A personal question to each member of your faculty, including prof. Harvey Goldstein and those who like his report:
12.17.19 12:54am - (3/3) What have YOU personally done to fight BDS activities on YOUR campus? Have you too decided that hostilities connected with Israel are kosher hostilities, not affecting students like me who are, or are perceived to be, proud of Israel? Wasn't it YOUR academic duty to act?

12.16.19 11:55pm - (Replying to @calabi_and_yau and @jeremyphoward) I listened to @jeremyphoward talk and kept asking myself: When was it given? It turned out it was given in 2012, before analysts realized that most of his aspirations can be algorithmatized. So, yes, it is related to #Bookofwhy

12.16.19 11:01pm - (Replying to @jouni_helske) Thanks for saving me an n-th attempt to understand what Gelman is talking about. It is not his fault, it is mainly mine. We live in two non-intersecting universes. I have solved a few toy problems, Gelman refuses to try even one. #Bookofwhy

12.16.19 10:56pm - (Replying to @calabi_and_yau) It would be of interest to document the various contexts in which business people yearn for "insight" and see if it confirms my theory that what they yearn for is "causal understanding", or at least some chunk of a causal model compatible with the data. #Bookofwhy

12.16.19 2:58pm - (Replying to @jouni_helske) I will comment on Gelman's post if you tell us why you find it to be an "interesting take." Or, how it could possibly change the way we/others are doing Causal Inference, however minutely. #Bookofwhy

12.16.19 4:28am - I consider myself progressive, so some academic colleagues call on me, resentful of Israel's courting right-wing politicians to pass anti-BDS laws. My answer to them: What have YOU personally done past 10 years to fight BDS activity on YOUR campus? Wasn't it YOUR academic duty?

12.16.19 3:34am - (Replying to @thosjleeper) We have considered, but S-admissible is reminiscent of backdoor admissible for identification. Moreover, it is almost 9 years since its first publication. What would you suggest? Reweighing-admissible"? perhaps. S-separator? "S-Equalizer"? Suggestions? #Bookofwhy

12.16.19 2:31am - (1/4) In view of the dominant role that re-weighing plays in extrapolating effects across populations, and the many Twitter requests for a concise graphical criterion that gives re-weighing its legitimacy, I am retweeting the criterion (called "S-admissibility"), in next 4 tweets.
12.16.19 2:31am - (2/4) It works on a selection diagram in which S nodes represent disparities between the target (*) population and study population (experimental). Z is a set of measurements. To test if Z is S-admissible (1) Remove all arrows pointing to X (2) Check if {X, Z} d-separates S from Y
12.16.19 2:31am - (3/4) If Z passes this test, then the reweighing formula is valid: P*(y|do(x)) = SUM_z P(y|do(x),z)P*(z) In words: Effect at target equals the Z-specific effects at study, averaged over Z, using the target distribution P*(z) as weight. Warning, this is merely a sufficient test.
12.16.19 2:31am - (4/4) Many more opportunities are available for situations in which a S-admissible set cannot be found. see and #Bookofwhy (And dont dismiss "trivial transportability," described in Graphs are fun! #Bookofwhy

12.16.19 1:50am - I am also unsure of what "rainfall became to Economics", but I am retweeting ONE sentence, because it is a GEM: "Mendelian Randomization is just another observational study, with a different set of assumptions." Quasi-experimentalists shoud keep this in mind. #Bookofwhyk

12.16.19 1:38am - (Replying to @davidsirota) You may be Jewish by birth, but that does not prevent you from conflating the issue, perhaps intentionally. Bernie Sanders is criticize for surrounding himself with Zionophobes like Linda Sarsour, consumed by genocidal agenda to eliminate Israel. Jews do not escape issues.

12.15.19 1:35pm - (Replying to @EpiEllie) Yes. (I thought I tweeted it before, but can't find it.)This criterion (called S-admissibility) is sufficient for transporting the effect by simple re-calibration on Z. But this is the simplest kind of transport. We can do much more even if such a Z is not measured.#Bookofwhy

12.15.19 1:23am - (Replying to @DrMikeH49 @markshiffer00 and @YousefMunayyer) Mike, thanks for using Zionophobia, our only fighting word which, for some reason, is still not "the ugliest word in town" but it's getting there. See my reasons for preferring it over anti-semitism:

12.15.19 12:45pm - (Replying to @EpiEllie) Which diagram specifically gives you hard time to decide if {X,Z} separates Y from S (in the revised DAG from which all arrows entering X are deleted)? This is a simple d-separation test -- duck soup in your Epi class. What's the problem? #Bookofwhy

12.15.19 12:21pm - (Replying to @YousefMunayyer) Agree! Equating anti-zionism with antisemitism is misguided; it gives the former a speck of legitimacy that racist ideologies do not deserve. Zionophobia is more dangerous on its own character and genocidal aims. Here is why: It's the ugliest word in town.

12.15.19 12:01pm - (Replying to @yudapearl and @wtgowers) This is another reason why I viewed Corbyn as a danger to humanity: Rabbi Sacks is a champion of universal values. I know him personally. We went together to Muslim schools in London and talked to kids on our common heritage. I weigh his word over Guardian

12.15.19 4:05am - (Replying to @Howard_Lovy and @lsarsour) Which proves that there IS a fundamental difference between "left-wing" and "right-wing" anti-Semitism. The latter is emotional and mother-milked, the former is calculated, strategic and eliminationist, ala @lsarsour .

12.15.19 3:50am - (Replying to @wtgowers) Admittedly, I was unaware of some these very concerning allegations. However, this report was prepared by dedicated scholars whom I know for years. In contrast, I know NONE of the academic signatories on the Guardian. Strange! I should know at least ONE.

12.15.19 3:21am - (Replying to @AndreaSaltelli @RonKenett and @fhuszar) I have read your Nature paper with great interest, yet it is not clear to me where "causal models" reside? In the statistical or mathematical category? My taxonomy of models (Ladder of Causation) is based on what language the assumptions must be expressed. Thoughts? #Bookofwhy

12.15.19 3:06am - (Replying to @RonKenett @AndreaSaltelli and @fhuszar) Pearl has this habit of celebrating what we can do today that we could not do yesterday. Ron has the habit of playing Solomon the wise: "Nothing is new under the sun" relevant or not. I'd go with Pearl; celebrations are fun and yield new results. Solomon doesn't #Bookofwhy

12.15.19 2:22am - (Replying to @james_r_lucas @roydanroy and 3 others) To predict a "wall" and overcome it, you need to have some idea of what's behind it. ML per se has no language to describe "what's behind". But see #Bookofwhy @tdietterich

12.15.19 1:59am - Little Jack Horner, sat in the corner... Oh What a Good Boy Am I. !!! Singing Oh What a merry land is England Singing Oh What a merry land is England

12.15.19 12:51am - (Replying to @mendel_random @stephensenn and 2 others) The #Bookofwhy unveils several appearances of causal notions in the periphery of statistical analysis, it even celebrates with great fanfare David Cox's (1958) explicit expression "quite unaffected". Those peripheral sparks can now be made explicit and unleashed. R U unhappy?

12.14.19 11:46pm - (Replying to @RonKenett and @AndreaSaltelli) I am celebrating a new ability: To take 10 meaningful examples and solve them formally from A to Z, and you are inviting me to go back to the hand-waving days of "gaining understanding" w/o solving any of those examples.What is gained by reminding people of those days? #Bookofwhy

12.14.19 11:05pm - (1/2) (Replying to @RonKenett and @AndreaSaltelli) I am open to be convinced about the existence of "different ways to generalise findings" as soon as you show me (not 'so and so', but YOU) how ONE of those "ways" decides which of the ten simple examples in is generalizable. #Bookofwhy
12.14.19 11:26pm - (2/2) (Replying to @yudapearl @RonKenett and @AndreaSaltelli) 2/2 Elias has compiled dozens of quotes on "different ways" and "so and so", from Campbell etal, to David Cox, to Manski. I don't see why the belief that "there r different ways" gives you more comfort than my conclusion (10 yr exploration) that there r'nt such ways. #Bookofwhy

12.14.19 8:21pm - (1/ ) Good question: "So how do researchers generalize experimental results if they don't know selection-diagrams or, worse, if they (eg econ.) don't even use DAGs?" Ans. They dont. They publish tons of paper with "generalizing" in the titles but, inside, they assume the analyst can
12.14.19 9:11pm - (2/ ) do the S-admissibility test in her/his head and come out with some "conditional ignorability" expression that justifies the result they want. Bingo! They did it! Reminiscent of the way folks did adjustments before DAGs. In we decry this culture in
12.14.19 9:11pm - (3/3) no ambiguous terms. But to those in the cultural bubble, decrying amounts to non-existing. And to economists, it may even amount to: "Don't show our students." Why the resistance? Selection diagrams highlight the needs to start with what we know - too hard for PO #Bookofwhy

12.14.19 7:52pm - I echo Democritus (~400 BC) who said: "I would rather find one cause than be a King of Persia". My bit: "You learn more by solving ONE toy problem than watching 10 panels"- even when everyone tells you "it ain't REAL-DATA" or "its textbook econ." It ain't! Try one! #Bookofwhy

12.14.19 6:54pm - (Replying to @EpiEllie) As they say in the Mishna "A child that does't ask, lead him to a question" (At Ptach Lo). So, which of the following can you read as "S-admissible"? (i.e., the x->y effect is transportable by reweighing on Z) #Bookofwhy

12.14.19 6:42pm - (Replying to @vatsal_maru and @BernieSanders) Poor Bernie wants us to believe he is a "man of principles," but is appointing advisers (eg Sarsour) that are consumed by unprincipled agenda. Why? Because other advisers tell him "this is what young voters go for" -- an unprincipled "principle", proven a disaster by Corbyn.

12.14.19 6:21pm - (Replying to @paulpharoah) You are absolutely right to demand English, but you need to say more than just "gibberish". Tell us where you get stuck.

12.14.19 6:18pm - (Replying to @nathankallus @SusanMurphylab1 and @Susan_Athey) My take: (1) “read the stats literature!” but only after taking "causal inference 101", eg (2) “pay SOME attention to (3) “read the empirical literature! but only after (1)&(2) (4) Solve one "toy problem" in CI, say from #Bookofwhy

12.14.19 3:00pm - Seriously. What did the panelists think when the words "the ML community's approach to causal inference" were spoken? Did they take the word "approach" to mean "aspirations" or "buzz words" or "pay attention to" or (my favorite) "careful thinking" #Bookofwhy

12.14.19 2:42pm - (Replying to @analisereal @EpiEllie and 2 others) The Mishna says: "The shy cannot learn and the strict cannot teach". We are facing a 5-yr old that is too shy to ask, and teachers that are open to any question. Will the shy help with: "I would like to know how...?" or "What do we do in case..." or... or... #Bookofwhy

12.14.19 2:30pm - Great thread! As an outside observer, the punchiest punch-line was: Contrast traditions that start with *the data generating process* versus those starting with *the algorithm*. The most perplexed line was "the ML community's *APPROACH* (???) to causal inference" #Bookofwhy

12.14.19 1:59pm - (Replying to @EpiEllie) What's the problem? Constructing a selection diagram? Or reading it to determine how to transport things? We can go over it one at a time, assuming, of course, that your 5-yr old can read DAGs. & while we are at it, other readers can enjoy its power on favorite probls.#Bookoofwhy

12.14.19 9:39am - (1/ ) Before parting from the beauty of selection diagrams and the S-admissibility criterion, it is important to empower @EpiEllie 5-yr olds with the proper historical perspective. In the same way that backdoor holds the key to the adjustment formula P(y|do(x)) = SUM_z P(y|x,z)P(z)
12.14.19 9:39am - (2/ ) which was used informally since Yule (1899), so does S-admissibility hold the key to the re-calibration formula (or re-stratification, or re-weighing, or re-standardization) P*(y|do(x)) = SUM_z P(y|do(x),z) P*(z) which was used informally by 18th century demographers
12.14.19 9:39am - (3/3) to transport mortality rates across populations (from PI to PI*). We make this historical connection in footnote 13 of and where readers can find additional magics of the graphical S-admissibility criterion. Enjoy. #bookofwhy

12.14.19 2:41am - A new paper deserving our attention, on alternative CI methods: My first reaction: Not clear why triangulating IV with backdoor methods should give us more information than, say, one backdoor method with two distinct admissible sets. #Bookofwhy

12.14.19 1:02am - Sincere apologies to my colleagues in the dpt of Archaeology at UCLA. It is the dpt of Anthropology that deserves the shame, as described here and here: Sorry for the confusion.

12.14.19 12:29am - Corbyn to Pellosi: Nancy, they all lied to us; about the angry mob, the shifting voting base, the radical millenials of Omar, Tlaib and JVP --they are just not there! Voters can't stand liberal politicians when they betray liberal values.

12.13.19 10:16pm - (Replying to @omaclaren) Where can we read more about Fraser and Bunke and what they related the fiducial argument for?

12.13.19 7:13pm - (Replying to @charleendadams) And my only unhappiness with Corbyn's defeat is that (according to analysts) he was not defeated for the reason he should have been: Racism, but for being wishy-washy on Brexit. Still, his defeat removes fears of shame & darkness for all my colleagues in the UK - Congratulations!

12.13.19 4:35pm - Personally, I have NOT encountered antisemitism, as confessed here But my students HAVE, especially in dpts of History and Archaeology, shamefully mentioned in It is the duty of every professor to expose those pockets of hate.

12.13.19 10:46am - (Replying to @charleendadams and @pablo_gps) University of Bristol!!! What a coincidence!!! I believe George @mendel_random is a professor there, and I presume he spends hours fighting the phenomenon you are describing. As I am doing in my university. Our reputation is at stakes, not to speak of our moral commitments.

12.13.19 10:23am - (Replying to @mendel_random and @pablo_gps) The understatement of the year: "Corbyn should have apologised for the behaviour of a smallish number of people mascerading as labour supporters." Compare to a comprehensive, documented report, by professionals that I know personally: "Smallish Number"!!!

12.13.19 1:37am - (Replying to @pablo_gps) The disaster I feared was not Labor (I loved Blair), nor Brexit. I feared seeing a racist like Corbyn as a PM of a European country that would be turned into a safe haven for Hamas and other anti-Western movements.

12.13.19 1:24am - This is the normal controversy we find in the literature. Then we have the added dimension of causality, see, and the fact that Bayes was chasing causes. #Bookofwhy

12.13.19 12:37am - (Replying to @ulrichspeck On the nail. Hot-headed students hijacked the volume on social media, and our politicians believe their voting base is really shifting. Look at what Linda Sarsour has done to Sanders and what Omar and Tlaib are doing to Pelosi.

12.13.19 12:21am - (Replying to @pablo_gps) Upon facing a sure disaster and an unknown, you dont "choose" the unknown but you sure escape the disaster.

12.13.19 12:16am - The pleasure was mutual and I was thrilled by the "philosophical twist" of today's talk. First time I speak about Bayes' pool table (p. 98 #Bookofwhy) realizing that most people (including Bayesians) dont really know what Bayes did and what the controversy is all about. New book?

12.12.19 11:19pm - On such historical events, my grandpa used to bless: Oh God! Hagomel Chassadim Tovim L'Amo Israel. As a secular Jew, but one who does believe in miracles, all my lips can say: Oh God! please give me the illusion that my Tweet has swayed at least one voter to defeat this racist!

12.12.19 9:40pm - (1/ ) (Replying to @EpiEllie) If DAG rules take a 5-year old, then selection takes a 6-year old. First, a tiny correction! Only a tiny fraction of DAG rules (ie, backdoor) is taught to 5-yr. The entire spectrum of identification opportunities (ie, do-calculus) awaits to be embraced in Epitweet #Bookofwhy
12.12.19 10:08pm - (2/ ) (Replying to @yudapearl and @EpiEllie) You are probably hoping for a backdoor rule when you say "5-year old". Don't worry, we've one for your 5-yr old in transportability. It is Def-8 & Cor-1 in If Y can be separated from S in G_X, Bingo! The effect is transportable! (by Eq 3.1) #Bookofwhy
12.12.19 10:19pm - (3/3) (Replying to @yudapearl and @EpiEllie) You see, It's duck-soup easy!! Even to a 5-yr old. All it takes is a simple language to say how populations differ, and a deep sense of belief that, if things are done the hard way in PO, they can be done the easy way in our natural language. #Bookofwhy

12.12.19 9:26pm - (Replying to @EpiEllie) The new name reminds users that the diagram stands for TWO populations, not one, and that the selection nodes [S] just switch from one population to another. This in mind,anyone who understand DAGs should understand 2-DAGs, ie, S-diagrams, as Elias shows .

12.12.19 7:27pm - (Replying to @EpiEllie) If you are happy with the functions behind the arrows, just follows the beautiful slides by Carlos and Elias, here: and tell me if/where their explanation of selection diagrams is less than simple. We will continue from there. Is it a deal? #Bookofwhy

12.12.19 3:15am - (Replying to @eddericu @EpiEllie and 2 others) This are great slides indeed, but they still require one to understand that behind the DAG's arrows there are FUNCTIONS and, not less important, that writing y=f(x,u*) is another way of writing y=g(x,u), with g neq f. #Bookofwhy

12.12.19 1:20am - My My! I almost forgot I did this podcast. Now the counterfactuals are beginning to play tricks on you: "You should have answered it this way, or that way." No, I shouldn't; we know that free will is just an illusion #Bookofwhy

12.11.19 11:00pm - (Replying to @yudapearl and @Corey_Yanofsky) But, for havens sake, even if p-Jews (People-hood defined Jews) are just a minority (they are a vast majority) aren't they entitled to dignity and respect like other p-minorities? Especially by the Office of (don't smile) "Equity, Diversity, Inclusion" at my university, UCLA.?

12.11.19 10:09pm - (Replying to @EpiEllie) I've tried to get inspired by your Dags Cartoons, to explain *selection diagrams* but, OOPS, I got stuck on agreeing about what a DAG is. "A Research Tool"? True. But a DAG also represents a collection of functions that defines a population. Can we agree, to continue? #Bookofwhy

12.11.19 9:28pm - (Replying to @EpiEllie) You know that I am the last person to "change topic" on Tweeters. For me, a selection diagram is just a DAG's way of representing two populations that differ in some of their underlying mechanisms (ie, equations). If you know DAGs, you know selection diagrams. #Bookofwhy

12.11.19 8:55pm - (Replying to @Corey_Yanofsky) And Ruth said: Your people will be my people, and your God will be my God. People-hood first, religion second.

12.11.19 7:05pm - I am retweeting, because this question comes up again and again: Why are diagrams "natural", "transparent" and "communicable"? Why? It also touches on the sanctity of RCT and the obsession with "well defined intervention" that some brilliant scholars refuse to abandon #Bookofwhy

12.11.19 6:51pm - (Replying to @EpiEllie) For ordinary folks who represent knowledge the way it is stored in our brain, in terms of FACTORS influencing one another, who have learned (eg. #Bookofwhy or that Potential Outcomes are DERIVED properties of that knowledge, not its elementary particles

12.11.19 5:13pm - Which reminds me. Tomorrow I'll be speaking to History of Science Reading Group (3pm UCLA Royce 314) where I will resurrect Bayes, Galton, Pearson, Wright and Haavelmo and ask again: Where have all the causes gone? Will Haavelmo's legacy survive in Econometrics ?#Bookofwhy

12.11.19 5:01pm - Remember Ander's "Russian Roulette" and all the arguments why one needs to use cross-world cryptic like P(Y(1)|Y(0)) to generalize experimental results? Well, Carlos and I took another look, Good news: You can do better in common language. #Bookofwhy.

12.11.19 4:37pm - It is a victory for BDS and an indictment of American universities that it should take a Presidential Order to define "Jewishness" the same way as Jews are defining themselves today, and the same way they have been defining themselves since the exodus from Egypt.

12.11.19 4:14pm - (Replying to @mstephens999 and @y2silence) Sure, because "salah" is not the only factor that determines the outcome of the game. Recall, however, a "unit" u, in both SCM and in PO stands for the sum total of all factors necessary to make the outcome 0 or 1, leaving nothing to uncertainty, thus no Pr(*). #Bookofwhy

12.11.19 3:50pm - (Replying to @mstephens999 and @y2silence) What makes "u" different than "i" ? If there is no difference, and both are just "anyone", we can remove the index. But if "u" have an allergy to zinc or some specific experience with zinc, we say: Pr (cold|do(zinc), specific knowledge). Still no index. #Bookofwhy

12.11.19 5:11am - This thoroughly-documented report by the Simon Wiesenthal Center chases sleep from my eyes. I imagine UK slipping into Corbynism, what it would do to my people, to my former country, Israel, and to the moral compass of the civilized world. Extremely worried.

12.11.19 1:44am - (Replying to @mstephens999 and @y2silence) I haven't thought about what Pr(Y(u)=y | do(X(u)=x)) could mean. But now that I think about it, the answer would be either 1 or 0. Because the Y(u) takes some value, say y' , which is either equal to y or not. If u is picked up at random we get P(Y=y|do(X=x)) w/o u. #Bookofwhy

12.11.19 1:33am - (Replying to @patrickkloesel @ASSAMeeting and 2 others) If invited, I'll break my vacation and go to San Diego to remind economists how Haavelmo defined "economic model", as I started to do here:, and of all the new things they can do by restoring his vision. #Bookofwhy

12.10.19 12:08am - This is a powerful presentation of the message I am trying to send to ML folks: "You and I know that we must climb the ladder of causation at some point. Therefore, to be safe, let's look at the rungs before we climb, and surely before we make believe we are already up"#Bookofwhy

12.9.19 9:52pm - (Replying to @mstephens999 and @y2silence) I see it as an advantage, to be able to distinguish between questions that can be answered by intervention, do-expressions, and those that cannot, because they are Rung-3 counterfactuals. Look how important this distinction is: , #Bookofwhy

12.9.19 4:31pm - (Replying to @mstephens999 and @y2silence) Do-expressions are probabilisitic, involving populations. To go to individual level, we go to SCM (Stractural Causal Model), where U=u is the individual. All counterfactuals, PO, ignorability etc etc can be derived from a fully specified SCM.(Including "consistency")#Bookofwhy

12.9.19 3:22pm - (Replying to @VartanKA @haldaume3 and 3 others) Attempts to synthesize have been successful. All is unified now and we fully understand where each concept stands in the scheme of things. eg. "ignorability" is logically implied by 'backdoor." No "synthesis" in Mostly Harmless where graphs mean disloyalty.#Bookofwhy

12.9.19 3:10pm - (Replying to @mstephens999 and @y2silence) When we write P(y|do(x), z) it means the full distribution of Y (not "mean") for every intervention do(X=x), for every z-specific class of individuals. It is so heterogeneous that calling it "heterogeneous" only diminishes its heterogeneity. #Bookofwhy

12.9.19 4:09pm - This is one merit of social networks: Reviewers and editors are no longer operating with impunity, they know that history will judge them, albeit not personally. They do not want to shame their Journal or their Alma Mater. #Bookofwhy

12.9.19 1:25am - (Replying to @RonKenett) Recall that transportability theory covers statistical transportability as well. So, what exactly changed that needs to be "transferred"? Population? Environment? distribution? #Bookofwhy

12.9.19 12:45am - (Replying to @RonKenett) The fact that they do not deal with causal effects does not make it "much wider". What does? Can you name a problem area (not an algorithm) that makes the scope of this survey "wider"? #Bookofwhy

12.9.19 12:27am - Where did you say this "editor" got his education? The quote is a vivid example of how research is impeded by those who slap careless labels on other people works (yesterday it was NPSEM-IE). As to content, I have not met a more heterogeneous object than a DAG. #Bookofwhy

12.8.19 11:30pm - I was also surprised to find the context worse. Modern statisticians are so embarrassed by his text that some resist: No, he wan't one of the founders of statistics. Some even told me: "Well, he was a socialist." Which proves that dogmas & politics still rule science. #Bookofwhy

12.8.19 5:24pm - (Replying to @AsraNomani) When Linda Sarsour says a Zionist can’t be a feminist....#zioness #WW84 #GalGadot, she is just confessing to being a FAKE feminist (see unfit to lead any progressive movement. When will Bernie see the harm she is causing him?

12.8.19 4:29pm - (Replying to @AndersHuitfeldt) Many people (unnamed) interpret the term "independent errors" (IE) to mean "Causal Markov condition", which is not just "semantics", but a severe restriction. This, IMHO, is a "gross misunderstanding," perhaps (?) unintended by the authors of the term "IE". #Bookofwhy

12.8.19 4:04pm - (Replying to @AndersHuitfeldt) The "misunderstanding" is on the part of readers who take "IE" as a restriction on SCM. Giving researchers the OPTION to express independence should not be construed as a restriction, it is LIBERATION when compared to systems that do not provide such option. #Bookofwhy

12.8.19 3:33pm - (Replying to @AndersHuitfeldt) That's a gross misunderstanding. The Causal Markov condition is NOT assumed to hold a priori in nonparametric SCM. Labeling it NPSEM-IE (IE ="independent errors") is misleading. A notational system that gives you the OPTION to assume IE should not be labeled IE. #Bookofwhy

12.8.19 3:08pm - (Replying to @hazqiyal) The bi-directed arrows denote correlated errors for some reasons, e.g., latent variables with names, un-named latents, selection bias, etc. If you suspect correlation, use them. Absence of those bi-directed arrows implies error-un-correlatedness for any reason. #Bookofwhy

12.8.19 8:06am - (Replying to @hazqiyal) Almost every diagram in Causality (2000) has bidirected arrows, X< - - ->Y, which stand for correlation between eps_X and eps_Y. #Bookofwhy

12.8.19 3:03am - "Transfer learning" (another word for "transportability") is what called my attention to this paper The title: "Learning subject-specific causal effects" made me doubly curious: Is this possible? Isn't "class-specific" the most we can hope for?#Bookofwhy

12.8.19 2:09am - Causal discovery enthusiasts among our readers should be interested in this symbiotic "discover-identify" approach, just posted on our website: #Bookofwhy

12.7.19 11:41pm - (Replying to @steppenjiff) Corbyn cannot be excused as lacking the brain to foresee the genocidal consequences of his words and actions. Do you genuinely believe the call for "from the river to the sea" is anything less than genocidal?

12.7.19 10:27pm - Agree with @GaryMarcus on the insufficiency of causal inference. But the "type-token distinction" is IMHO well formalized in structural models as "unit level vs. population level distinction" (Causality p. 310). The firing-squad example in #Bookofwhy shows token counterfactuals.

12.7.19 8:28pm - (Replying to @jon_y_huang) Thanks for posting @diedre_tobias . And I am happy to see that Jamie and I agree on at least one thing: “We are in the midst of a Causal Revolution!". Our disagreements dwarf when we remember what it took to get here and the huge work still ahead of us. #Bookofwhy

12.7.19 8:01pm - Luciana Berger is my UK heroine, for standing up to Corbynism.

12.7.19 7:44pm - (Replying to @aminjorati) "voices are not heard?" You must be kidding! Come to UCLA, my campus, where Palestinian victim-hood is celebrated day and night with megaphones and marching bands, and where BDS activists chant: “We don't want two states, we want 1948!" You must be kidding!

12.7.19 7:01pm - (Replying to @peder_isager @dingding_peng and 3 others) In cyclic equations, the d-separation may breakdown when non-linearities are introduced. [It holds for linear cyclic systems]. For "Identifying Independencies in Causal Graphs with Feedback" see #Bookofwhy

12.7.19 6:49pm - "Zionophobic" is the only word I use for Zionophobes -- a much uglier word than "Anti-Semite". The latter may often be a victim of misguided upbringing. The former has no excuse, it's a cold & calculated call for genocide.

12.7.19 6:30pm - There are (at least) two inaccuracies in this paper:(1) Nonparametric SEM does not assume error-independence (see<-->) therefore, no one should use the acronym NPSEM-IE. (2) NPSEM does NOT assume that all variables (V ) can be intervened on. See #Bookofwhy

12.7.19 12:13pm - (Replying to @wtgowers) She will always find a reason, or a paragraph, or an adjective to prevent her from voting for a two-state solution. Why? Such a vote would turn her into a Zionist, thus losing her voting base. Watch and see how next time she says: It does't include the "right of return".

12.7.19 5:30am - (Replying to @Kaniska_Mohanty) I do recommend Primer. Chapters are available here, and it is a really good book!!! #Bookofwhy

12.7.19 5:26am - Omar, Tlaib, AOC & Pressley -- the four elephants in the room. American law makers have not internalized a simple fact: Zionophobes cannot utter the words "two-states solution" and Israel's neighbors suffer from the same speech impairment.

12.6.19 11:00pm - (Replying to @geomblog @L_badikho and 4 others) These people are wonderful, very broad, and technically superb. But all the King's horses and all the King's men can't turn a mathematical impossibility into a possibility. "Fairness" requires acknowledging certain impossibilities, else all King's men would be wasted.#Bookofwhy

12.6.19 1:20am - (Replying to @markcannon5 @AnimaAnandkumar and 5 others) I hope you are not suggesting that DL is beyond enrichment, or that DL+CI symbiosis is not a compelling way of overcoming DL's shortcomings. see #Bookofwhy

12.6.19 1:06am - Continuing this track on Bernie Sanders & Linda Sarsour marriage. Poor Bernie thought Zionophobes could be kept under leash for some semblance of progressive values. Now he is wondering who owns the leash, who's "surrogate" to whom?

12.5.19 11:23pm - (Replying to @L_badikho @timnitGebru and 6 others) Agree totally. Logic too originated from moral disputations. At the same time we should not ignore the fact that current ML proposals for dealing with "fairness" are dominated by a model-blind thinking, while "fairness" demands a model-based approach. #Bookofwhy.

12.5.19 11:08pm - (Replying to @L_badikho @timnitGebru and 2 others) I was explicit about "expand their expertise" which is different from "lack technical expertise". The former is respectful and hopeful, so am I. #Bookofwhy

12.5.19 10:51pm - Very well put. I am seeing this "ignorance above all" phenomenon first hand at UCLA, Department of Archaeology. I, too, cast the conflict as a morality play, a simple choice between co-existence and elimination. A very simple choice.

12.5.19 10:13pm - (Replying to @timelessdev) CI stands for "Causal Inference" as defined and described in or

12.5.19 10:06pm - Agree! And the question is: Are ML folks flexible enough to understand that "fairness" demands expansion of their expertise? Or will they insist on handling it the model-blind way, and waste the funds and trust that society now pours on them. Same to "explainability #Bookofwhy

12.5.19 7:20pm - "Fairness" involves CI issues such as mediation (ie distinction between direct and indirect effects), undesirable side effects, and protected variables. It is hard to see how these issues can be managed without a causal model of reality. #Bookofwhys

12.5.19 7:14pm - Why are "fairness" and ML so often mentioned in the same breath? True, the former can be compromised by irresponsible practices of the latter, but this does not mean that the tools available to ML folks are adequate for preventing those compromises. #Bookofwhy

12.5.19 7:01pm - A glimpse at the NeurIPS 2019 program shows that CI is well represented. IBM, for example, seems to be taking the DL+CI symbiosis quite seriously: I fail to see though why "fairness" and ML are so often mentioned in the same breath. #Bookofwhy

12.5.19 6:05pm - I can try and contribute a couple of papers, but I am afraid the title will invite "what computers cannot do" type of speculating philosophers (eg Penrose, Searle) with whom I do not wish to be identified. #Bookofwhy

12.5.19 2:41pm - (Replying to @IzaTabaro) Bernie can't decided between the Sarsourish crowd that flocks his band wagon and the principles he learned at home.

12.5.19 1:57pm - Viewing history from a causal lens is an exciting adventure for me. I will be giving this talk to the History & Philosophy of Science Group, at UCLA. December 12, 3 pm, Royce Hall Room 314. If you happen to be around that day, just drop in. #Bookofwhy

12.5.19 6:36am - (Replying to @BlakeFlayton) On the contrary. She just confessed to being an incurable racist, hence unfit in any progressive movement. Poor Bernie, choosing "surrogates" turns out more hazardous than one thinks. Has he given up on attracting non-racist voters. ???

12.4.19 11:29pm - Linda Sarsour (Bernie Sanders' surrogate) is my muse this week. Friday she said: "Israel is based on supremacy," Yesterday, she apologized: "[No, No]...not the Jewish people. I apologize for the confusion." I had to tweet her: Sister Linda! Sister Linda!

12.4.19 7:24pm - (Replying to @lsarsour) Sister Linda, Sister Linda! A decent way to take back what you said is to say what you meant: e.g., "Zionism is NOT what I said! It's a noble dream, a people's homecoming journey, one we should revere, respect and assist --That's what I meant." Then apologize for the confusion.

12.4.19 5:33pm - (Replying to @RabbiWolpe) Here is the context: "I love Jews, and Bernie is my proof, but I was bought up to hate, and Zionophobia is safe, and so much chic"

12.3.19 10:03pm - I have thoroughly enjoyed re-reading @EinatWilf 's article a day after Linda Sarsour, the darling of Fake Feminism (see, tried again to contaminate Zionism with her racist thoughts.

12.3.19 12:56pm - (Replying to @EpiEllie) I do not recall any paper rejected for having been pre-printed. But I may be too young to recall all cases, I'll check with my buddies on the Editorial Board.

12.3.19 8:20am - (Replying to @nickchk) This may be a new IV, but the graph-void presentation makes me commiserate with econ. readers who must make sense of cryptic sentences like: "an instrument correlated with the confounders, but which itself is not causally related to the direct effect of the treatment." #Bookofwhy

12.3.19 1:44am - (Replying to @DemMaj4Israel) "Israel’s built on idea that every people is entitled to self determination" - How true! But did you ask Bernie Sanders and Nancy Pelosi if they think about that truth when they choose advisers and spokespersons?.

12.3.19 1:17am - Being part of a condemned minority is not new to Jewish history, but being a PROUD member of such minority IS new and, strangely, you feel fortunate not sharing the embarrassment of the Sapiens delegates voting tomorrow, having to face their families the next day.

12.2.19 3:25am - (Replying to @juli_schuess @thosjleeper and 6 others) On first scan, this is a very valuable paper, connecting the huge survey-inference literature with causal graphs and showing how crucial design issues in the former are rooted in causal thinking and can now be formalized and made transparent in graphical models. Kudos! #Bookofwhy

12.2.19 1:29am - (Replying to @curtatkisson) I'm not familiar with Meng 2018, should I? And who are "they"? Sorry, I can't guess.

12.1.19 10:40pm - (Replying to @JDHaltigan) I rarely agree with Gelman but, in this case, he is right that the true effects can be higher or lower than what they found; the same holds under "natural experiments" which are just observational studies with different assumptions, see, #Bookofwhy

12.1.19 9:43pm - (Replying to @RonKenett) What is not clear to me about the "second dimension" in your paper is whether the "matching of data resolution/aggregation and analysis goal" is a desirable feature to have, or an algorithm that actually establishes this matching in some formal setting. #Bookofwhy

12.1.19 7:47pm - (Replying to @AviMayer) It's worse than pure antisemitism, its ugly Zionophobia. It's time we stop treating the latter as the lesser of the two evils. See And let's deny its legitimacy using its real name, Zionophobia, not antizionism, which sounds almost as innocent as anti-tax

12.1.19 4:55pm - Granularity in Scientific Explanation has been a favorite topic of discussion for many philosophers, but when cast in a specific representation (DAG), capable of producing explanations, it now assumes a new level of clarity: #Bookofwhy

12.1.19 4:16pm - A new paper on my desk (i.e., screen), calling for attention, combining linear or nonlinear conditional independence tests with a causal discovery algorithm to estimate causal networks from real-world data. #Bookofwhy

12.1.19 3:22pm - (Replying to @brianoflondon @EinatWilf and 2 others) Fairly gloomy picture, I've suspected this kind of defeatism for quite some time. Any healthy organization trying to pressure the @BoardofDeputies into some level of pride and conviction?

12.1.19 8:21am - (Replying to @yudapearl @XiXiDu and 2 others) And too often they treat Antizionism as a semi-acceptable form of racism, less dangerous than Antisemitism. It isn't.

12.1.19 7:53am - (Replying to @XiXiDu @EinatWilf and @HananyaNaftali) Part of the reason Antizionism has metastasized among the political left is that OUR leaders and OUR spokespersons so rarely challenge its legitimacy and barely expose its moral stench.

12.1.19 7:39am - For heaven's sake, why is everyone questioning Corbyn on antisemitism, a question he is so skillful in deflecting. Why not charge him squarely with incurable Zionophobia, a moral deformity he cannot possibly hide. @EinatWilf @HananyaNaftali

12.1.19 1:31am - (Replying to @MaccormickIan and @ewenharrison) This is a step-change given current approaches. I agree. And one way toward making this step-change is outlined here #Bookofwhy. If there are others, bring them along.

12.1.19 1:13am - (Replying to @ArcusCoTangens @bariweiss and 3 others) Sure there is: The latter is twice uglier and much more dangerous:

12.1.19 12:04am - The litmus question our spokespersons refrain from asking presidential candidates: "Are you prepared to state that, as president, you will not support anyone running for office who denies Jews the right to self-determination in their historical homeland?" So, I am asking instead.

11.30.19 10:26pm - (Replying to @bariweiss @EinatWilf and 2 others) Senator Sanders, Fighting antisemitism requires courage, not populist talking-points. Are you ready for the Pearl's litmus test: "As President, I will not support anyone running for office who denies Jews the right to self determination in their historical homeland." Answer=?

11.30.19 8:35pm - I should not stay silent on this day, Nov. 30, marking the expulsion of Jews from Arab countries. My wife was one of them, and her story is documented here in an interview for the Shoah Foundation

11.30.19 1:42pm - I now realize that ML folks must be going through the same trauma. Armed with Logic and Probability, the two most powerful languages in science, they find it hard to believe that they cannot climb from Rung-1 to Rung-2 (&-3) in the Ladder of Causation. It's a scandal. #Bookofwhy

11.30.19 1:09pm - (Replying to @prem_k and @GaryMarcus) This was one of the most traumatic realization of my life, that probability, my mother tongue, and the mother tongue of statistical sciences cannot deal with cause & effect, the elementary particles of human thought. I do not blame statisticians for chocking over the #Bookofwhy

11.30.19 5:45am - (Replying to @fdabl) This is a nice summary of CI at its core. Thank you @fdabl for introducing the subject to a wider audience in such friendly and commonsensical way. You proved my point that causality is simple if not purposely mystified #Bookofwhy

11.30.19 4:06am - Surprise!! #Bookofwhy is #1 bestseller in Biostatistics. Reasoning statistically, if Bio statisticians take it seriously, chances are statisticians too are not irredeemable.

11.30.19 3:56am - (Replying to @MarkusSchacher) A standard causal DAG presumes unknown (and unrepresented) factors u to affect every node in the graph. The firing squad example, however, does not allow such factors, to demonstrate how counterfactuals are computed in deterministic systems. #Bookofwhy

11.30.19 3:49am - (Replying to @yudapearl @omaclaren and 10 others) Correction. The query is Q: "There exists a units u s.t. Y(x,u)>Y(x',u)" for some x and x' in the domain of X." The post at estimates the percentage of such units u. #Bookofwhy

11.30.19 3:19am - (Replying to @MarkusSchacher) You are absolutely right. All of them should be iff. "D only if A or B" asserts that we rule out other causes of death, except (possibly) A or B". Thanks for pointing this out. #Bookofwhy

11.30.19 3:13am - (Replying to @bschoelkopf and @ylecun) What Festschrift? I am not going to retire. I still have 37 productive years (Jewish legend). And someone must convince the remaining 90% of academia that students should take CI-101 before stat-101. Thanks for a great paper. #Bookofwhy

11.30.19 2:57am - (Replying to @omaclaren @Lester_Domes and 9 others) From seeing just one Y(x) we cannot conclude that "X causes Y". To conclude, we need to define a new Boolean query Q: "There exist two units, u and u' s.t. Y(x,u)>Y(x,u')" , and then estimate E[Q] . For one way of estimating E[Q], see #Bookofwhy

11.30.19 2:33am - (Replying to @yudapearl @omaclaren and 10 others) I expect the TT crowd to hit the roof: "Yes, but how do you estimate E[Y(x=1)] without an RCT?" Me: "Relax, the question was whether E[Y(x=1)] would satisfy me, not how I estimate it. I can do the estimation too (see #Bookofwhy), but lets not conflate two separate questions."

11.30.19 2:23am - (Replying to @yudapearl @omaclaren and 10 others) Fisher, "You really want to "compare" the new yield to last year's yield, don't you?" Me: "Never mind what I do with that information, it's my business, just tell me E[Y(x=1)]". Fisher quits, and I remain offering 100 Euro to any statistician who can predict E[Y(x=1)]. #Bookofwhy

11.30.19 2:11am - (Replying to @omaclaren @Lester_Domes and 9 others) In some cases I would be happy with just E[Y(x)]. If I am a farmer in Rothhamstead 1924, and Fisher came to me with his RCT, I'd say: Just tell me what Yield I'd get if I use the new Fertilizer on the entire field. I dont want to Latin Square my field, nor to randomize. Aha! Says

11.30.19 1:31am - Here is another offering to Lady History in the wake of November 29th, laden with facts and driven by a question: What kind of Palestine would our Lady wish to see in the Middle East?

11.30.19 12:53am - To close the gap between Big Data and Causality, a panel will be conducted at the IM DATA conference in Pasadena, where 3 UCLA graduates will show how CI can make data interpretation robust and meaningful See details:, #Bookofwhy

11.29.19 11:18pm - To clarify, what is a "Causal Notion"? It's a notion that cannot be defined in terms of distributions over observed variables, so Y(x) is certainly "causal". Harvard's folks would object: "If you find E[Y(x)]=5, it doesn't tell you if X causes Y." True RCT thinking. #Bookofwhy

11.29.19 10:59pm - (Replying to @omaclaren @Lester_Domes and 9 others) If you are asking me whether I ultimately want a function Y(x) -- not necessarily; in some problems I would be happy with just E[Y(23)/Y(7)]. In SCM you get the whole function for free. In Harvard you can't talk about Y(x) w/o some Y(x') to feel part of the RCT culture.#Bookofwhy

11.29.19 10:47pm - (Replying to @omaclaren @Lester_Domes and 9 others) Change in variance of Y(x) means that there is at least one individual (labeled u, in Y=f(x,u)) for whom the change is between Y=0 and Y=1. Quite dramatic. #Bookofwhy

11.29.19 10:42pm - (Replying to @omaclaren @Lester_Domes and 9 others) I can't really speak for what the others want. I know that many expressed suspicion of a creature named Y(x), or even E[Y(x)], because (my conjecture) Harvard's graduates are trained, born and conceived in RCT, all else is suspect. TT fashion is part of this culture #Bookofwhy

11.29.19 7:12pm - (Replying to @kaz_yos) Thanks for posting. I was not aware of this Cookie blog before. And I am always surprised to see articles on missing data containing no graphs. How do the authors check their assumptions? Do they really believe in them? Beyond me! #Bookofwhy

11.29.19 6:59pm - (Replying to @omaclaren @Lester_Domes and 9 others) There is a subtlety regarding the object of discussion. Is it the object Y(x)? Or the exercise of estimating Y(x) or the estimate E[Y(x)])? Do we have a established standard criterion as to what is "causal" and what is not? I have proposed a crisp criterion. Objection?#Bookofwhy

11.29.19 3:19pm - (Replying to @omaclaren @Lester_Domes and 9 others) The expression "X causes Y" is used in scientific conversations, and it often stands for the formal condition "Y(x) is NOT constant in x." But some people use it more narrowly, to mean E[Y(x)] is not constant, or even E[Y(1)]>E[Y(0)]. #Bookofwhy

11.29.19 2:21pm - (Replying to @omaclaren @Lester_Domes and 9 others) If I, as the author of the DAG, know that Y(X=x) is a constant, then I have the license to remove the arrow. But putting an arrow there will not count as "model misspecification". It will count only as under-utilizing one's knowledge. Recall, "May be" is not knowledge #Bookofwhy

11.29.19 1:50pm - (Replying to @Lester_Domes @EpiEllie and 8 others) Great! No pathology. So we agree that it is OK to call the function f(x,y) = P(Y=y| do(X=x)) "causal effect of X on Y", though in some extreme cases it may be independent on x. And despite the fact that some people may confuse it with the difference E(x=1, y)-E(x=0,y).#Bookofwhy

11.29.19 5:18am - (Replying to @Lester_Domes @EpiEllie and 8 others) Moreover, a null-effect is still a kind of "effect", same as a flat-response being some kind of a "response", and a "constant function" is still a function, albeit pathological. #Bookofwhy.

11.29.19 4:01am - (Replying to @Kweku_OA) @Kweku_OA I somehow overlooked your question, sorry. The best introduction is Pirkei Avot, which stands for "Essays of our Fathers," a collection of ancient Tweets (100-200 AD) on principles of ethical behavior Enjoy.

11.29.19 3:04am - (Replying to @elderofziyon) I usually add one more item: Final ownership to be determined by Palestinians readiness to spell the word "co-existence".

11.29.19 2:53am - (Replying to @MarkusSchacher) Consider the Firing Squad example in #Bookofwhy page 39. Bidirected logical implication exists between any two variables. Yes causal implication goes only along the arrows.

11.29.19 2:49am - From Deep Learning to "Deep Feeling: AI and Emotions." This mind-blowing exhibition should be of interest to our artistic readers: Petach Tikvah is not far, just 10 km from Tel Aviv. Founded in 1878 by 5 orthodox Jewish cowboys. Worth a visit. #Bookofwhy

11.28.19 9:35pm - (Replying to @MarkusSchacher) No relation. Logical implication is invariant under contra-position, not so causal implication. #Bookofwhy

11.28.19 5:48am - (1/ ) Wishing all readers a joyous and meaningful Thanksgiving weekend, and adding a personal note for Friday, November 29. I have a secret contract with Lady History: She forgives my moments of weakness and I remember her moments of grace. One such moment was November 29, 1947
11.28.19 5:48am - (2/ ) when I, my family, our neighbors and the whole town heard the voice from the radio: "Yugoslavia abstained! The resolution [UN 181] has passed!" And suddenly, a terrifying shout, 80 generations of age, came out of the throats of everyone in town, including pets, fish, and
11.28.19 5:48am - (3/3) babies in their cribs: WE HAVE A STATE !!!!!!!! To fulfill my obligation to Lady History, I am sharing a 2018 oped that further describes the significance of that moment: Happy Thanksgiving!

11.27.19 11:40pm - (Replying to @ConiByera @EpiEllie and 8 others) Not only who are "we"?. What are the "counterfactual framework?".

11.27.19 8:22pm - (Replying to @ConiByera @LucyStats and 8 others) What I called "causal effect" (in Causality 2000) was the entire sequence: Y(X=x) x=1, 2, 3, ... or E[Y(X=x)] x=1,2,3, .... From which you can form differences eg E[Y(X=5)]-E[Y(X=3)] or play with ratios, eg E[Y(X=2)]/E[Y(X=1)], or follow your fancy. #Bookofwhy

11.27.19 3:27pm - (Replying to @EpiEllie) Eureka !!! Eureka!! With the help of another colleague I now understand what Target Trial (TT) is! And I also understand why no TT expert could explain it to me. It is really a "framework", a misguided one, but a "framework". I will soon post what I've found. Eureka! #Bookofwhy

11.27.19 8:08am - (Replying to @robertwplatt @rohitpojha and 5 others) I cannot work through these question in the TT framework, because no one would tell me whether TT constrains me, and how. Perhaps someone versed in TT can do it and tell us if there is anything wrong in assuming no constraints. IOW TT permits us to forget about TT. #Bookofwhy

11.27.19 7:58am - (Replying to @LucyStats @AlexaYakubovich and 7 others) I can't believe this is the official position of TT. Are you saying E[Y(X=0)] and E[Y(X=1)] may not be causal, and the difference E[Y(X=0)] - E[Y(X=1)] is causal just because we have a comparison? We need a top TT expert to confirm. You see why I am less than enthused? #Bookofwhy

11.27.19 7:49am - (Replying to @robertwplatt @rohitpojha and 5 others) I read the literature and I acknowledge that many find it helpful as a way of thinking. The answer I do not get is "What is it?" New or old, what is the idea? Is each of my aspirin questions legitimate though it invokes no comparison, no trial and no randomization? #Bookofwhy

11.27.19 7:38am - (Replying to @AlexaYakubovich @LucyStats and 7 others) I did not say they aren't, I merely asked the adherents of TT to tell us if each of those questions IN ISOLATION, each invoking no comparison, no trial and no randomization, would be a legitimate question to ask. Simple question. #Bookofwhy

11.27.19 7:02am - (Replying to @LucyStats @AnjaLeist and 6 others) Before I decide what to do with the predictions under 1 tablet of aspirin, 2 tablets, 0 tablets etc etc, I want to make sure that each of these predictions is a legitimate question to ask. (Regardless of how we establish them). Q:Are they legitimate questions under TT? #Bookofwhy

11.27.19 5:46am - (1/ ) (Replying to @AnjaLeist @rohitpojha and 5 others) You are the first to tell me what "target trial" is, thanks! But you are imposing a severe restriction on the kind of questions we can ask, forbidding me from asking for the probability that my headache will subside in 1 hour if I, given my medical history etc, decide to take
11.27.19 6:08am - (2/ ) (Replying to @yudapearl @AnjaLeist and 6 others) an aspirin, just because my question invokes NO COMPARISON. I am talking about ASKING, not about establishing the answer. I now look with great compasion at the oppressed adherents of the "target trial" framework, forbidden from asking the questions they really wish to ask and
11.27.19 6:08am - (2/2) (Replying to @yudapearl @AnjaLeist and 6 others) forced to ask only questions that someone put in their mouth. I can't join this constraining culture; would you? And all for the sake of emulating RCT? I can't. A good framework is one that allows researchers to articulate what they truly wish to know. #Bookofwhy

11.27.19 5:11am - (Replying to @luketrailrunner) My indicators would be: (1) Explicit mention of commitment to causality research and acknowledgement of its crucial role, in the Institute's Statement of Purpose. (2) Inclusion of genuine CI researchers on the institute's Advisory Board. Without the two, we're back to statistics.

11.27.19 4:59am - (Replying to @rohitpojha @EpiEllie and 4 others) No objection! I am genuinely trying to understand what it is. Since it is marketed and hailed as a "new framework," I fear to be missing something by not adopting it. Am I? Or can I and my students relax back to the framework that kept us happy and productive thus far? #Bookofwhy

11.27.19 3:29am - (Replying to @rohitpojha @EpiEllie and 4 others) We agree on what is needed to predict the consequences of taking an aspirin, we are both causal inference experts, so we have no differences here. My humble question is "What would we lose if we purge the phrase "target trial" entirely from our language?" What? #Bookofwhy

11.27.19 3:12am - (Replying to @_JakeHumphries @EpiEllie and 5 others) I am using metaphors when I wish to describe someone who is committed to consistently violate the very essence of the "target trial" philosophy, and I humbly ask "Where would this someone go wrong?". Well, where? #Bookofwhy

11.27.19 3:04am - (Replying to @_JakeHumphries @EpiEllie and 5 others) I need analysis (and data) to estimate the probability that my headache will subside next half hour if I take aspirin. Note: I did not mention "trial". I will mention "trial" if I use RCT to interrogate nature. But I would never use "target trial" in the same breath. #Bookofwhy

11.27.19 2:43am - (Replying to @EpiEllie @rohitpojha and 4 others) I will bend to this nuance and talk only about my target being "an intent to implement a policy". Where would I go wrong viewing this intent to be my TARGET, not a TRIAL, vowing never to mention TRIAL when I define what I wish to estimate. Would such heresy hurt me? #Bookofwhy

11.27.19 1:30am - (Replying to @EpiEllie @rohitpojha and 4 others) The podcast appears useful to many in the audience. But humble me still wonder what error would I make if I continue to view RCT as in #Bookofwhy, ie, a means to interrogate nature, not a target in itself, and I view my target to be a policy, not a trial. Where would I go wrong?

11.26.19 11:11pm - (Replying to @rohitpojha @AbbyCScience and 2 others) I have asked epidemiologists colleagues if they knew what "target trial" is. They smiled and admitted "not a clue". Can you explain it to the uninitiated? What if one's ultimate goal is not a "target trial" but a "target policy? Would anything be missed? #Bookofwhy

11.26.19 10:00pm - I am retweeting this reply because of its general concern to many CI researchers. Public funds are pouring to many "Data Science Institutes" which are likely to continue the exclusive "data-centric" focus of DL/ML unless an explicit commitment is made to snap out of it.#Bookofwhy

11.26.19 5:28pm - (Replying to @ylecun and @bschoelkopf) I see dozens of "Data Science Institutes" erected across the country, I read their manifestos and I check their advisory boards. Causality does not seem to be on their agenda. Which makes one doubt whether the Ladder has been internalized and where this hype will end. #Bookofwhy

11.26.19 4:24pm - Now that we all know how to compute counterfactuals (e.g.,, we can take a step back and enjoy Silvia Karasu's article (Psychology Today) on the poetic side of Counterfactual Thinking and Imagining: #Bookofwhy

11.26.19 3:56pm - (Replying to @EricTopol) Thanks for the kind words, Eric. I am usually against humility in science, because so much CAN be accomplished. My point, however, is that to climb the Ladder one has to look at the Rungs. #Bookofwhy

11.26.19 2:43pm - There's much truth to what you're saying. The idea that there are theoretical impediments to ML methods is hard for ML folks to internalize.And repeated assurances that causal inference is just one aspect of what ML has been doing all along do not encourage them to try.#Bookofwhy

11.26.19 1:41pm - (Replying to @ArielOrtizBobea and @causalinf) This is fair, with two additional touches. (1) among those "complex relationships" rests this one: "is the effect estimable by any new method?" which DAGs now automates. (2) "intractable" practically means "undoable", like coloring a map with 3 colors. #Bookofwhy

11.26.19 4:51am - A very comprehensive, delightful and inspiring paper. Recommended to ALL, not just MANY ML/AI folks. Note also that @bschoelkopf does not perceive me as "polarizing the field" as suggested by @ylecun here: SCM unifies, invites and educates. #Bookofwhy

11.25.19 9:53pm - (Replying to @ArielOrtizBobea and @causalinf) Ed Leamer asked me the same question. Take your favorite algebraic representation, say 3 structural eqs., and ask your students if it has any testable implication, or, which parameter is estimable by OLS, or identified by some IV. Its doable, yes, but have they been asked before?

11.25.19 9:40pm - (Replying to @causalinf and @ArielOrtizBobea) Indeed, visualization is only one tiny feature of DAGs. The feature that is least appreciated by economists is computation. "So what" some say, "It takes me longer, but I'll get it!" Wrong! Intractability induces helplessness! They just wont try! They try only tractable problems.

11.25.19 6:59pm - (Replying to @EpiEllie) I've never met a "clinician", I only heard about them. What epi principle are they missing?

11.25.19 1:47pm - (Replying to @ChrisCrandall16 @joel_saarelma and 3 others) My point is that the arguments that convinced psych departments to include stat classes in their programs (1/2 century ago) are doubly more compelling when applied to causal reasoning classes. #Bookofwhy

11.25.19 1:37pm - (Replying to @osazuwa @ang_hermann and 3 others) Conditional Policies and Stochastic Policies are discussed in Chapter 4 of Causality. It shows that if we know P(y|do(x), z) then we can compute the effect of a policy X=g(z) i.e., Act X in response to observation Z=z. See also Dynamic Process Control p. 74. #Bookofwhy

11.25.19 1:18pm - (Replying to @learnfromerror) I've read what Spanos writes on Simpson's paradox. But do you, as a philosopher of statistics find it satisfactory? e.g., sufficient for determining the correct action in any of the examples discussed here ??? #Bookofwhy

11.25.19 5:44am - (Replying to @joel_saarelma @ChrisCrandall16 and 3 others) PRIMER is taught in high schools, Are psych students less deserving? Introduction to causal inference is recommended before stat 101. #Bookofwhy

11.25.19 3:13am - (Replying to @RonKenett) Let's start with the virtues of automation as a tactical effort, then, if we are lucky and succeed to emulate Ron Kenett on a computer, I would not reject the extra benefit, i.e., understanding scientific thought, understanding ourselves, and finding a cure for cancer.#Bookofwhy

11.25.19 3:05am - (Replying to @RamiEmad1988 and @JustinTrudeau) "Not for the civilians" !!!. The credibility of anyone reporting from the Middle East hinges upon one simple litmus test: Willingness to ascribe some human qualities to Israelis, however feeble. Zionophobes never do.

11.25.19 1:27am - (Replying to @RonKenett) The goal of automation forces you to think about your thinking as an object demanding understanding at the input-output level. Informal thinking, however decorated, "critical" or not, is still thinking "from within". #Bookofwhy

11.25.19 12:21am - (Replying to @RonKenett) Aspirations to formalize then automate human thinking did more for our understanding of human thinking than all the thinking that was done before. We might never get there, but the fall outs of trying far exceed the dead-ends of old-fashion "thinking". #Bookofwhy

11.24.19 7:40pm - (Replying to @mdubowitz) Reform Khomeini-style makes me crave for Reza Shah.

11.24.19 9:25pm - (Replying to @learnfromerror) This has been my observation too, but isn't it the same when statisticians say "this is a subject matter issue", thus avoiding a position on which claim is warranted and which is not. Recall how statisticians have been avoiding Simpson's paradox #Bookofwhy

11.24.19 7:28pm - (Replying to @joe_miller0 @_pierreblanc and 8 others) Great question. The functions used in the estimation stage (say a regression line) approximate a population. Those that operate behind the arrows govern each individual unit. The finite sample accuracy of the former can be quantified, not so the latter.See Cslty p. 35. #Bookofwhy

11.24.19 5:12pm - (Replying to @joe_miller0 @_pierreblanc and 8 others) Or with finite data combined with a ML wizard.

11.24.19 5:06pm - (Replying to @joe_miller0 @_pierreblanc and 8 others) Given the X--->Y graph, no functions, we get P(y|x) and P(y|do(x)) from the data. We need the function y=f(x, eps) if we want Y_x (eps). #Bookofwhy

11.24.19 4:49pm - @NicoleBarbaro . A most generous compliment, thanks! I only wish economists start to view themselves as social scientists. #Bookofwhy

11.24.19 4:04pm - (Replying to @RamiEmad1988 and @JustinTrudeau) I wrote: "The truth about what Israel has done for Syrian refugees", not Zionophobic propaganda.

11.24.19 4:00pm - (Replying to @joe_miller0 @_pierreblanc and 8 others) Depending on what "causal model" we have. The graph in itself (with data) can only give us interventions. We need the functions behind the arrows to get countefactuals. It is neatly explained and exemplified here: #Bookofwhy

11.24.19 2:52pm - Twitter is a powerful provoker of sound bites. I wish this one gets inscribed on my tomb: "We need to see the Rungs before we climb the Ladder" #Bookofwhy

11.24.19 2:43pm - (Replying to @prem_k and @Atmaprajnananda) It might have some connection, but it is too spiritual for me. I am an input-output poet. #Bookofwhy

11.23.19 7:03pm - (Replying to @JakeSearcy) how incredibly useful curve fitting can be, and how incredibly easy it is to see its limits.

11.23.19 4:47pm - (Replying to @dileeplearning) Do we have a Turing test for 'symbolism"?

11.23.19 2:40pm - What I am saying about ML/AI and "curve fitting" is not really aggressive, nor is it my saying; it is an innocent message sent to us from the mathematics of cause and effect. We need to see its Rungs before we climb the Ladder, and it is on p.28 of #Bookofwhy

11.23.19 6:09am - (1/ ) This is a key insight. This mysterious place holder got different names in different disciplines. Statisticians called it "subject matter information", some used "Bayesian priors." Economists called it "economic theory", etc. etc., Tell us more, about what pushed philosophers
11.23.19 6:09am - (2/ ) to postulate this animal. It comes up again and again, what was suppressed into namelessness turns out to be none other but a causal model, mathematizable, well understood, and definitely not mysterious. #Bookofwhy

11.23.19 2:07pm - (Replying to @Ace_Harsh) Kindly explain what you mean by "relative".

11.23.19 2:01am - (Replying to @JustinTrudeau) Respectfully, denunciation is what racists love to hear. The way to neutralize this mob is for your office to echo the message that this speaker came to tell York University: the truth about what Israel has done for Syrian refugees. They fear truth, not denunciations.

11.22.19 10:16pm - (Replying to @adamdedwards and @Yu_Ke__) If customers of difference races enter the bank at random, then "race" becomes a "random variable" in the eyes of the bank teller. "Interaction" does not make it "complicated" or "metaphysical". #Bookofwhy

11.22.19 10:08pm - (Replying to @analisereal) The analogy is not perfect. When you use a slide-rule the assumptions are as clear as English, .... #Bookofwhy

11.22.19 2:35pm - (Replying to @Yu_Ke__ and @adamdedwards) Suppose you apply for a loan in a Nazi-own bank and then you complain: "Why was my application denied?" Wouldn't the answer: "Because you are not Arian" be an honest answer? (albeit not exactly "fair") #Bookofwhy

11.22.19 3:34am - I remember another quote related to Anderson's "correlation supersedes causation". It goes: "Correlation can get us to Babylon, but not to Athens". I used it here: . #Bookofwhy

11.22.19 3:15am - (Replying to @Yu_Ke__ and @adamdedwards) Being a member of one race or another surely CAUSES racially biased people to act differently. Why deny them that honor? #Bookofwhy

11.22.19 12:51am - (Replying to @ERMANigeria) I am not sure what Anderson meant by "supersedes" but, if we judge by correlation-based statistics, it hasn't advanced much since Pearson if we count the number of regression analysts who can resolve Simpson's paradox. eg., #Bookofwhy.

11.22.19 12:15am - As I read this post today, 5 years after, I am amazed at how little things have changed in the eco-bubble. Glad this Twitter post helps grad students in economics see what they need to rebel against. #Bookofwhy

11.21.19 11:07pm - Sharing an interesting paper: The need for context-specific independencies emerged in the 1980's to supplement d-separation. This paper extends it to causal inference using annotated graphs; it seems more transparent than specifying PO's. #Bookofwhy

11.21.19 10:40pm - (Replying to @ma_macneil @rlmcelreath and 2 others) I would not use the phrase "causal salad" because it connotes a complex and hotly contested topic. It isn't. Causal Inference is embarrassingly simple and coherent, despite some who are trying to mystify it. Dont let them. #Bookofwhy

11.21.19 10:23pm - (Replying to @Yu_Ke__ and @adamdedwards) Race has effects, same with earthquake. And these effects exist "beyond correlation". #Bookofwhy

11.21.19 10:03pm - (1/ ) (Replying to @JessGeraldYoung @WhitneyEpi and 9 others) 1/ I would separate conceptual understanding of mediation from its identification. After all we would only labor to identify something if it is meaningful once identified. My favorite conceptualization is: "The fraction of cases OWED to mediation" (necessity) and, likewise,
11.21.19 10:07pm - (2/ ) (Replying to @yudapearl @JessGeraldYoung and 10 others) 2/ "The fraction EXPLAINED BY mediation" (sufficiency). Both are defined, discussed and exemplified here Now, given that our scientific knowledge (SCM) assigns values to ALL counterfactuals, I do not see any impediment to conceptualize mediation in terms
11.21.19 10:17pm - (3/3) (Replying to @yudapearl @JessGeraldYoung and 10 others) of nested counterfactuals - identification comes later. And if my meaningful effect is not identified, I would approximate, but I would not sacrifice meaning on the alter of identification. @Bookofwhy ps. Check, it's a good paper, obscured by Wiley

11.21.19 12:35pm - Interesting connections between tractable versions of Bayesian networks and causal diagrams, despite the somewhat negative conclusions. #Bookofwhy

11.21.19 3:32am - Gee, where did I get the time in July to answer Quora type questions? But, in retrospect, it's a good answer, which has withstood the test of time. Glad to share it on Twitter. #Bookofwhy

11.21.19 1:49am - I am not a health scientist, nor am I around NYC today, but I would surely love to hear the latest in "causal data science". It sounds like a new building Columbia is constructing. #Bookofwhy

11.20.19 9:13pm - (Replying to @stewarthu and @mark_vdlaan) We do not have strict guidelines on this distinction. Our main criterion is: Would other readers be able to learn some lesson (or a tool, or an impossibility) that would help their next research problem. #Bookofwhy

11.20.19 8:42pm - The enthusiastic interest in Rebane's Polytree Algorithm (118 likes) reminds me of another beautiful discovery algorithm. Suppose you have a tree, and observe only its leaf-nodes. Can you discover the internal-structure of the tree? A miracle! #Bookofwhy

11.20.19 8:26pm - Please welcome @mark_vdlaan who has just joined our education channel. Mark is a friend of commonsense, and a co-founder of the Journal of Causal Inference, which is about to become a serious competitor of Econometrika (for reasons we have discussed here yesterday) #Bookofwhy

11.20.19 7:55pm - Fully agree! The do-operator operates on variables in the context of a causal model of reality. The recipient of the operator can be a state (eg temperature, race) or manipulable event (eg. take drug). In I explain the importance of the former. #Bookofwhy

11.20.19 5:35pm - (Replying to @renatrigiorese) Agree. "Deep associationism" is a more accurate term. #Bookofwhy

11.20.19 3:53pm - (Replying to @DaniCMBelg @lgmoneda and 3 others) From what I can see, the paper by Mason etal is not using DAGs and, as a result, the assumptions are opaque and theoretical guarantees are absent. By DAGs I mean a model of the reasons for missingness, as is described here: . #Bookofwhy

11.19.19 11:00pm - Still another historic day: Our Twitter counter tells me that this station has reached 25K followers. Not bad for an education channel that started 17 months ago, June 27, 2018. I thank all readers for keeping me awake, and for giving me the illusion of being useful. #Bookofwhy

11.19.19 10:45pm - Another historical moment I must retweet, since I am old enough to vividly remember Sadat speech at the Israeli Knesset in 1977, see, it gave my countrymen an existence proof to hope that, one day, a Palestinian leader will have such courage.

11.19.19 10:24pm - (Replying to @wagonomics and @amitabhchandra2) Interesting theory, but if we read carefully what economists write about regression (see eg I am not sure the "long CI traditions" have helped them much, especially these days, when those traditions are trashed the "natural experimentalists" #Bookofwhy

11.19.19 9:58pm - (Replying to @wagonomics and @amitabhchandra2) Only history will judge if econ. editorial leadership is endowed with the kind of insight you describe: "to see important papers using a new tool and become convinced a review paper will be impactful". My job is to alert them to existing gaps and growing insularity. #Bookofwhy

11.19.19 9:23pm - A proverb one can't help but retweet. I only wish it comes true in our lifetime, not like other Biblical promises.

11.19.19 7:20pm - (Replying to @yudapearl @wagonomics and @amitabhchandra2) And, moreover, "pressuring" the editorial establishment is simply not in econ. culture; academic survival means tribal allegiance, not "pressure". #Bookofwhy

11.19.19 7:14pm - (Replying to @wagonomics and @amitabhchandra2) I hold fellow economists to a higher esteem. I know they do not need a minimum-wage example to recognize the importance, in their own research, of mediation analysis and external validity. Yet it would not occur to them that other fields are already 10 years ahead in these tasks.

11.19.19 7:00pm - (Replying to @AlexAlvPerez) I will never get tired recommending Primer. It is essentially available free, here Enjoy! #Bookofwhy

11.19.19 4:45pm - @amitabhchandra2 What you are saying is very encouraging to me. Still, as an Editor, I would feel it ain't right if my readers have to wait 10-20 years for an amazon book to keep up with the latest. I wonder if other editors are aware of how econ. is perceived. #Bookofwhy.

11.19.19 4:01pm - (Replying to @chrisalbon and @databozo) Recall, every structural model confers an unambiguous truth value to every conceivable counterfactual. Need a ruling? Unveil your model. #Bookofwhy

11.19.19 1:35pm - Discovering Polytrees. Rebane's algorithm (1987) was the first attempt to uncover causal structures from raw data. I was gratified today to see that it was extended to include hidden variables: #Bookofwhy

11.19.19 5:13am - (Replying to @elderofziyon) Settlements are not THE obstacle to peace, because there is a ten time taller obstacle there. Yet by diverting attention from the taller one they make it harder to remove, or even to point to.

11.19.19 4:46am - (Replying to @mikejohansenmd @amitabhchandra2 and @Kweku_OA) Highly accessible review articles are available in all disciplines (eg except economics. We know for a fact that economists do not read such articles unless they are home blessed, and NBER is an exclusive club. #Bookofwhya

11.19.19 4:15am - (Replying to @amitabhchandra2 and @Kweku_OA) I am more concerned about the fact that in the past half a century the econometric literature has not invited/published any REVIEW ARTICLE on graphical models, a methodology that other disciplines have adopted as a second language. This would worry me as an Editor. #Bookofwhy

11.19.19 3:28am - (Replying to @Kweku_OA) Thanks for bestowing hope onto economics. I am still waiting to hear from @amitabhchandra2 (former Editor of RES) if he agrees that editors should assume greater leadership in de-insulating their respective fields. #Bookofwhy

11.19.19 2:11am - (Replying to @arinbasu and @UCNZ) Spoiling the young is what makes book-writing such a thrill.

11.19.19 1:53am - (Replying to @StephStammel) Or, at the very least, not spoil it with statistics.

11.18.19 6:21pm - Hillarious! I can't forgive my mother for denying me such Bayes'ic rights! But I took revenge in this cartoon:

11.18.19 6:04pm - (Replying to @EinatWilf) As much as I think Israel’s settlement project is its greatest folly, this much is true: People have a right to a land to the extent they bestow that right to their neighbors. And that works both ways.

11.18.19 2:02pm - (Replying to @deboerk07) This is indeed interesting, the first "innovation" that came to her mind was about causal inference. She will soon see more innovationis, now that @eliasbareinboim joined Columbia. #Bookofwhy

11.18.19 4:01am - Sport tends to make miracles: First time I hear the Israeli anthem (Hatikva = "The Hope") played by Arab musicians, sending shivers in my spine. It's fusing two hopes: (1) "to be a free nation" and (2) to make peace with our neighbors. The first is 2,000 yrs old, the second 71.

11.18.19 3:41am - An interesting new paper just crossed my screen:, connecting graphical models to machine learning routines in an economic journal. A word of caution: The black-box is not really black, it is back-door admissible. #Bookofwhy

11.17.19 8:52pm - For our Argentinian scholars and international soccer fans, a rare picture of Leo Messi landing in Israel, from a Jordanian Airline plane, despite rockets from Gaza and in defiance of BDS cacophony. More soccer may be the treatment needed and good will the effect modifier.

11.17.19 3:37pm - (Replying to @stephensenn @RonKenett and 2 others) For AI-ers, "comes from experience" is not a sufficient answer, because experiential knowledge must reside in one's mind to be usable. Notice that, taking your description literally, a regression model is all we need to automate your thoughts. Not sure you would agree. #Bookofwhy

11.17.19 3:26pm - (Replying to @yudapearl @RonKenett and 3 others) So, my next project is to read Cox (1958). Give me a week to see if I can translate Cox's hints to encodable knowledge. This conversation reveals a basic disconnect between "designers" and AI-ers. The former THINK, the latter WORRY: Can I automate my thoughts? #Bookofwhy

11.17.19 3:16pm - (Replying to @RonKenett @f2harrell and 2 others) I am not referring to re-randomization. I am referring to the knowledge that makes @stephensenn smarter than a novice or a coin flipper. I am unable to extract from him the nature of that knowledge but, U'r right, Cox has lots of comments about where this knowledge comes from.

11.17.19 2:26pm - (Replying to @RonKenett @f2harrell and 2 others) Of course blocking and randomization are different. Still, compare an expert designer who assigns blocking scientifically to a novice, who assigns things thoughtlessly, say by flipping a coin. If the former does better, why? What does he have? Knowledge?Calculator?DAG? #Bookofwhy

11.17.19 2:15pm - (Replying to @stephensenn @f2harrell and 2 others) The reason I was changing from "hair color" to "eye color" was to guess the general criterion for choosing a blocking factor. But if it is so simple:"CONSIDERED PREDICTIVE" we can automate it, all we need is a model of predictiveness." Now you will say NO & I'm back to guessing

11.17.19 1:53pm - (Replying to @RonKenett @f2harrell and 2 others) If human researchers can assign blocking better than a random coin, and they have no more than a mind-DAG to guide them, then the answer to your question is "Of course!". If they use more than a mind-DAG, what is the extra ingredient? Lets capture it on a computer. #Bookofwhy

11.17.19 7:32am - (Replying to @f2harrell @stephensenn and 2 others) @f2harrell . If you understand "experimental design" (I don't) perhaps you can help me unveil the mental process by which @stephensenn decides, prior to seeing data, that "age" will be a better block than say "eye color" for reducing error in predicting heart attacks. #Bookofwhy

11.17.19 5:31am - (Replying to @stephensenn @RonKenett and @learnfromerror) Whats the point of challenging DAGs to do this or that when I am trying to understand what this or that is? I am coming to you as a student, and you take a defensive position as if I was a threat. I am interested only in the mental process behind the choice of blocks.#Bookofwhy

11.17.19 5:22am - (Replying to @yudapearl @RonKenett and 2 others) I am trying to learn what this mental process is, so that you and me can decide if it is within statistics on not. So, please be patient with me and help me understand this mental process. Lo Hakapdan Melamed (Hebrew for: "The uptight makes a bad teacher"). #Bookofwhy

11.17.19 5:02am - (Replying to @RonKenett @stephensenn and @learnfromerror) I did not say "statistics does not have the tools to handle blocking." I said the phrase that you used: "this is one of the important areas of statistics" was also used about causality, since 1975. And I now understand that it meant: "statistics cannot handle causality" (cont.)
11.17.19 5:07am - (Replying to @yudapearl @RonKenett and 2 others) which was true, it did not have the language. Whether statistics has the tools to handle blocking depends on whether the mental process by which you decide (prior to examining the data) that one factor is a better block than another can be articulated in statistical language.

11.17.19 2:44am - For the many readers of our education channel, this new post introduces Fréchet inequalities using modern visualization techniques and discusses their applications and their fascinating history. Enjoy, and thank @smueller for the movies. #Bookofwhy

11.17.19 2:04am - (Replying to @RonKenett @stephensenn and @learnfromerror) Funny. I have heard the phrase " one of the important areas of statistics" reiterated since 1975, about causality. And I now understand that it meant: "Sorry, statistics does not have the tools to handle it." Nowadays, with new tools, I hope to capture your intuition. #Bookofwhy

11.17.19 1:48am - (Replying to @stephensenn @RonKenett and @learnfromerror) This is precisely what we, causal inference students, are trying to learn from you. I, for one, do not understand the vocabulary of "design", "block" "within subject studies" "inherently predictive" etc. But once we learn your intuition, we will automate it, promise. #Bookofwhy

11.17.19 1:37am - (Replying to @stephensenn @RonKenett and @learnfromerror) Is "inherently predictive" different from "predictive" or "highly associated with"? Recall, this judgement is made BEFORE we collect any data, so it cannot be defined circularly as e.g., "choose whatever improves precision". #Bookofwhy

11.17.19 1:28am - (Replying to @RonKenett @stephensenn and @learnfromerror) Your paragraph makes students (me) believe that you need to try out both "age" and "hair colors" on the data before deciding. But you surely advise them not to waste time trying out every possible measurement. What intuition guides you prior to taking data? #Bookofwhy

11.17.19 1:16am - (Replying to @RonKenett @stephensenn and @learnfromerror) I look back at my Tweet and find nothing there about causality, just an innocent question: "what makes 'age' a better block than say 'hair color'?" . How do you explain to students in class, BEFORE seeing any data, that the former would be better for estimation. #Bookofwhy

11.16.19 10:39pm - (Replying to @RonKenett @stephensenn and @learnfromerror) @RonKenett . Why is it that design experts tend to refer me to authorities when I ask "what makes 'age' a better block than say 'hair color'" Why cant I find one to tell me what a 'good block' means to him/her PERSONALLY, or how they explain it to their students? Shy? #Bookofwhy

11.16.19 9:24pm - Another thing that would help us get out of the "well-defined intervention rut" is to recall how the rut came about. In 1974 Don Rubin defined PO in the context of "treatment assignment", which appealed to statisticians and epidemiologists. In 1973, however, David Lewis ...
11.16.19 9:24pm - defined counterfactuals (ie, PO) in terms of "alternative worlds", a metaphysical conception that only philosophers could buy. The two conceptions are unified in SCM ( but neither field managed to cut its umbilical cord. Glad U'R "getting out". #Bookofwhy

11.16.19 7:19pm - Thanks for the original text. I was quoting from memory, and was sure there was some reference to "Rabbi". But "honor" serves the same purpose in our secular world. "Who is honored? He who honors his fellow human beings".(Ben Zoma).

11.16.19 6:28pm - (Replying to @ProfStiff and @marclamonthill) Marc Lamont Hill will continue to spew BDS-hate with impunity until someone dares call him a "Zionophobic Racist", which may perhaps convince him to look at himself in the mirror.

11.16.19 6:13pm - This Tweet helped me formulate an exciting agenda for philosophers of statistics: "Go over the statistical literature of the 20th century and explicate, formalize and algorithmatize key concepts which statisticians dismissed as "beliefs" "subjective judgement" or "too Bayesian"

11.16.19 6:01pm - Sharing a new refinement on "what is a cause", and "Is race a cause" and "why does it matter?" #Bookofwhy

11.16.19 2:57pm - (Replying to @JBlevins0 @RonKenett and 2 others) Beautifully stated: "approaches the experiment with some beliefs". The reason I am excited is that, nowadays, "beliefs" are not left to speculations but are being explicated, formalized and algorithmitized. So, what are the beliefs that make blocking improve precision? #Bookofwhy

11.16.19 2:40pm - (Replying to @JBlevins0 @RonKenett and 2 others) Unfortunately, I dont have Vol 1 of K&H. Can any design expert summarize, conceptually, what makes one block better than another, w/o referring to authority? In computer science we have no authorities, so we just confess: "For me, a good block is one which ...". #Bookofwhy

11.16.19 2:15pm - (Replying to @ben_golub @causalinf and 2 others) Oddly, my patience only ran thinner since turning 83. And Jewish tradition now gives me only 37 years of hope to see econometric catching up to modernity. Short time, considering the observed movement. #Bookofwhy

11.16.19 1:36pm - (Replying to @JBlevins0 @RonKenett and 2 others) All I could get my hands on was vol. 2 of H&K. Do you remember how they show (or justify) that "age" is a good block, while "street address" is not so good? Or, better yet, how does a good stat professor justify it in class? Thanks #Bookofwhy

11.16.19 1:06pm - (Replying to @JBlevins0 @RonKenett and 2 others) Thanks for pointing to this source which I am eager to explore. But the label "Bayesian problem" must be taken with a grain of salt, because statisticians tend to label any subjective judgement "Bayesian", even judgments about causation, see #Bookofwhy

11.16.19 12:54am - (Replying to @arinbasu) I am sorry you got stuck on a beautiful chapter, which some readers have found to make counterfactuals as clear as daylight. See if Section 4.4 of makes it clearer, it has more examples. #Bookofwhy

11.15.19 6:28am - (Replying to @stephensenn @RonKenett and @learnfromerror) I am trying to learn from you about experimental design. I am not comparing it to causal calculus, nor do I claim anything. I'm just trying humbly to learn how blocking was proved useful. Why not teach me? #Bookofwhy

11.15.19 6:18am - A LOADED question to @amitabhchandra2 . We have had a lively discussion here on Twitter whether economics is more insular than other disciplines. Do you think econ. editors are doing enough to expose readers to emerging methodologies that neighboring disciplines have found useful?

11.15.19 6:07am - I can't envision an AI PhD student that would miss this opportunity. #Bookofwhy

11.15.19 6:01am - (Replying to @stephensenn @RonKenett and @learnfromerror) May I assume that Nelder's "allocation map" is the notational device that I was searching for? "Reading" will be situated differently than "age", and he has the mathematics to tell us that "age" is a good block, while "reading" may be terrible. Did I get it right? #Bookofwhy

11.15.19 5:33am - (Replying to @stephensenn @RonKenett and @learnfromerror) I still can't see how John Nedler could distinguish a good block (say "age") from its proxy (say "reading level") lacking mathematical notation in 1965. Moreover, how could he possibly PROVE that blocking on age improves precision, when blocking on "reading" may not. #Bookofwhy

11.15.19 5:22am - A down-to-earth guide for the missing-data perplexed. Our tutorial has just been posted on arxiv: #Bookofwhy

11.15.19 3:07am - (Replying to @stephensenn @RonKenett and @learnfromerror) It is hard for me to believe that this intuition was made formal before 1975. Why? Because "determinant" is a causal notion, and statistics did not have the notation to distinguish a "determinant" from its proxies; the latter could be bad blocks. Thus my disbelief. #Bookofwhy

11.15.19 1:39am - (Replying to @RonKenett @stephensenn and @learnfromerror) I am stuck on something much simpler. What is "block"? Is it what you DECIDE to block, or what you SHOULD block? Wikipedia says: "Block=Where units are similar to each other". Do you buy it? I'm also looking for a proof that if you block on some B you would do better. #Bookofwhy

11.15.19 12:47am - (Replying to @stephensenn @RonKenett and @learnfromerror) I could not find the history of "block design" in Peter Goos writings. Does any reader know where block design was first proposed? And, more importantly, where it was first proven (mathematically) to improve precision? History galore! #Bookofwhy

11.15.19 12:20am - (Replying to @cryptoecongames @kareem_carr and @EpiEllie) A DAG/tool forces more than clarity. For example, it tells researchers if they chose a good covariate set for adjustment. As for "implementation"? Here it is: "Choose a good set and continue the same way as you did with the bad set." #Bookofwhy

11.15.19 12:07am - (Replying to @RonKenett @stephensenn and @learnfromerror) Ron - speaking of Lord's paradox, @stephensenn and @learnfromerror , do you happen to know where "block design" was first used? first defined? or first justified formally? #Bookofwhy

11.14.19 11:31pm - (Replying to @RonKenett) My point was/is that even what we took to be purely statistical tasks (eg running RCT, optimizing predictions, handling missing data) require causal knowledge, hence DAGs. Statisticians avoided DAGs successfully, because they used their mind-DAGs instead. Robots can't. #Bookofwhy

11.14.19 11:14pm - (Replying to @hoss_tbf and @doinkboy) 1. Are you sure this was a stats textbook? Mine refrained from discussing mediation, because it requires causal thinking & vocabulary - strongly tabooed. 2. Would you be surprised if some journals (JSEM, Psychometrika) still spoke the way your textbook did? #Bookofwhy

11.14.19 10:09pm - (Replying to @YoniMichanie and @HananyaNaftali) Beg to disagree, Palestinian education is a much greater obstacle to peace.

11.14.19 7:21pm - Even just to "go get reliable, replicable, experimental data..." we need to understand the problem domain, ie, its DAG. How else can we decide if "what worked here will also work there"? How else can we decide what nuisance factors to control for? No escape! #Bookofwhy

11.14.19 6:46pm - Nice exposition of transport methodology, unconflated by "target trials" and other RCT imitations. But what is a "g-comp estimator"? Is it a special estimator of the transport formula, or just any consistent estimator? #Bookofwhy

11.14.19 6:20pm - Thanks for re-posting and re-calling a quote: " tools that are indispensable in solving simple problems are unlikely to become dispensable when problems become more complex." Which explains why some researchers hate simple problems--they tend to expose one's tool set. #Bookofwhy

11.14.19 2:05am - (Replying to @jonasobleser @stephensenn and 3 others) I would love to weigh in but can someone explain (in Twitter friendly language) what "latent change score model" is? And how it is used by its proponents? Thanks #Bookofwhy

11.14.19 1:46am - (Replying to @kareem_carr) Can't help asking: Did you say that "classical statistics knows how to generalize to new data"? I am asking because I have learned that classical statistics has no clue on how to generalize, eg., Are we speaking same "classics"? #Bookofwhy

11.14.19 1:24am - (Replying to @Statisticko and @JuhaKarvanen) Sounds like a wild party that I should enjoy. I know @JuhaKarvanen since his 2014 paper on study design. Say hello for me. Now I know causality is in good hands. Thanks. #Bookofwhy

11.14.19 12:18am - These are the best lectures (and course notes) I have seen in statistical education. Not so much because they are DAGy, but b/c they are given by a "data scientist", as opposed to a "data analyst." We need a new discipline: "scientific statistics" to replace "statistical science"

11.13.19 9:54pm - (Replying to @HananyaNaftali) California

11.13.19 3:04pm - I am not worried about the woolly socks, my heart goes to "causality" being squeezed between two statisticians, top and bottom. Give her a chance!! Or, better yet, give them a chance to rethink what they are doing. #Bookofwhy

11.13.19 7:02am - Any connection to the missingness graphs used here: ??

11.13.19 6:30am - My hero! 6 years old Shoham. I wish I could tell him what I was told at my 6th birthday, that when I grow up there would be no wars in the world.

11.13.19 5:44am - Several readers inquired on the relations between Markov processes and structural models. This new paper translates continuous-time discrete-state Markov processes to structural equations for the purpose of managing actions and counterfactuals. #Bookofwhy

11.13.19 5:37am - (Replying to @lgmoneda @Khipu_AI and 3 others) Anyone analyzing missing data using DAGs must understand missing data. Kudos! #Bookofwhy

11.13.19 1:30am - Erika, had I not been inflicted with incurable modesty I would recommend the Primer much more strongly than I do. It is geared for non-statisticians, and it starts where intuition resides (not RCT) @smueller uses it for high school. The best! @Bookofwhy

11.12.19 10:46pm - (Replying to @GilTroy) "Fighting against hatred" is easy. Even hate-soaked Linda Sarsour says she fights hate. What counts is "fighting Zionophobia", because no living Zionophobe can control this pathological obsession.

11.12.19 8:59pm - (Replying to @jazchaz) Joe Biden made history, not by repeating what others say, but by the way he stood up to JVP hecklers: "Show me one Arab leader willing to accept Israel, and I will talk to you about the occupation" (paraphrased from a friendly memory).

11.12.19 7:18pm - To my brothers and sisters in Israel, I join all people of conscience in telling you that your resilience and struggle for life, peace and justice are inspirational to all of humanity.

11.12.19 4:04pm - (Replying to @JadePinkSameera) The merits of DAGs lie not in helping the "most logical" researchers, but in helping the lazy ones, those who want the arrows to do the thinking for them, and protect them from errors and over-generalizations. That's why I am surprised when cultures slip into darkness. #Bookofwhy

11.12.19 3:43pm - (Replying to @AlyssaHarlow) #epitwitter and @EpiEllie , I just tweeted this, because I see counter-examples to every statement on this thread. I think there is an endemic misconception here, about selection bias, that can be easily corrected with a quick glance at the DAG. #Bookofwhy

11.12.19 2:39pm - (Replying to @admiyoung and @AlyssaHarlow) More generally, questions about selection bias cannot be answered in the language of "associations" or "prediction", no matter how hard we try. We must consult a causal diagram and first find an unbiased "estimand", IPW is just an estimator. see #Bookofwhy

11.12.19 5:51am - (Replying to @hangingnoodles) Well put! Competent statisticians nowadays are grad students who return their textbooks to the publisher with a note: "Same old stuff?" #Bookofwhy @rorysutherland

11.12.19 3:15am - The siren sound that I hear from Israel today has the same pitch as the one I heard in 1948, when Egyptian war planes bombed Tel Aviv. But then, people did not say "Israel has a right to defend", it was self evident. Today, a nation must beg public opinion for the right to live.

11.12.19 2:06am - (Replying to @AdanZBecerra1 @UChicago and @WestPoint_USMA) Did students find it "super hard"? or "super fun"?

11.11.19 7:51pm - (Replying to @DrCatherineB) Did you say "super hard"?. The only complaints I have heard about causal inference were: "How come no one told us it's so easy?" Seriously, have you read any of the "for fun" literature? BTW, consistency is not an "assumption" but a simple corollary of counterfactuals. #Bookofwhy

11.11.19 7:36pm - Thank you @Carthica for re-posting this tutorial on graphical models. It should open many eyes to the simplicity of causal inference, as it did in 2012, when most people still did "belief updating." Note the 3 bullets on the last slide, how true! #Bookofwhy

11.11.19 4:54pm - (Replying to @GilTroy) For J Street to convince us that "pro-Israel pro-peace" are not empty words, it must commit at least 1% of its efforts to fighting Zionophobia. It does'nt. Thus, J Street may not be anti-Semitic in intent, but it sure aint "pro-Israel pro-peace". Is it, God forbids pro-deception?

11.11.19 4:28pm - The beauty of the graphical framework in is that all generalizations, including subset to superset, are managed by the same methodology. Life would be utterly impossible if we demand "a new approach" for every nuance of generalization. #bookofwhy

11.11.19 1:07pm - (Replying to @deaneckles and @juli_schuess) I think N. Cartwright was the first to use := . The notation Y(x) = a + bx + ε is not sufficient to define a structural equation. You need a structural model to define what Y(x) is. IOW, for y= a + bx + ε to be structural, X must be the only direct cause of Y. #Bookofwhy

11.11.19 4:11am - (Replying to @juli_schuess) Today we think it is "disturbing"; at the time it was standard thinking. Even today, what do we know about the thousands upon thousands of "regression analysts"? We do not know much about what they think, because they are silent. Dead silent. Scared to confess thoughts #Bookofwhy

11.11.19 1:47am - The thread itself, including Miguel's proposal, can be found here #Bookofwhy

11.11.19 1:24am - "How to extend causal inferences from a RTC to a target population?" I am retweeting this thread since many readers ask this question, and many know of the comprehensive solution described in and in using graphical models. 1/2 #Bookofwhy.
11.11.19 1:24am - I was surprised therefore to read Miguel's proposal to recast the whole question in the opaque language of PO, where assumptions are so far removed from scientific understanding. A face-to-face comparison of the two approaches is given here:

11.10.19 10:07pm - (1/2) (Replying to @_MiguelHernan @Biometrics_ibs and @EpidemiologyLWW) Miguel, I beg to strongly disagree with your proposed approach to generalizing empirical results. Translating issues and solutions from the language of graphical models to the language of potential outcomes blurs the issues and hides the results. Such translations suffer from 11.10.19 10:13pm - (2/2) (Replying to @yudapearl @_MiguelHernan and 2 others) three incurable shortcomings, summarized in 3 vivid bullets here:, also known as TPT (Transparency, Power and Testability). So, given that Epi folks already speak graphs, what's the point of retreating from clarity? #Bookofwhy

11.10.19 9:35pm - For mediation analysts among us, here is an interesting article that hit my screen today: I still havn't absorbed the semantics of the proposed Indirect Effect, but the fact that it generalizes the frontdoor tells me there's substance to it.#Bookofwhy

11.10.19 1:33pm - (1/2) It's an excellent paper, agree, that is rarely cited, because it got published in a "handbook", ie, books that only librarians read. The official publications of the SEM community, eg JSEM or Psychometrika, are still buried in confusions over what SEM is about, awaiting,
11.10.19 1:33pm - (2/2) like most econometric journals, for a change of guards, and a speedy catch up with modernity. The litmus test for modernity is Eq. (6), the definition of counterfactuals, of which most SEM folks are unaware, and which is the basis of most works in causal inference.#Bookofwhy

11.10.19 12:59am - For machines to generate human-level explanations, we should listen to what philosophers of language say about causative nuances in English. This paper say it in the language of SCM, so it should be accessible to readers of #Bookofwhy.

11.10.19 12:32am - (Replying to @tdietterich and @twimlai) I buy your main thesis that "understanding" is a matter of degree. Would you buy mine, that "understanding a domain" means answering 'what if' questions about hypothetical scenarios in that domain? #Bookofwhy

11.9.19 9:10pm - (Replying to @_MiguelHernan @MariaGlymour and @HarvardEpi) And I am waiting eagerly for the day the teaching curiculum @HarvardEpi will be revamped forward, to the #targetQuery approach, with the #TargetTrial approach being one way of answering queries of interest, including queries about counterfactuals and explanations. #Bookofwhy

11.9.19 8:46pm - (Replying to @tdietterich and @twimlai) Thanks.

11.9.19 7:25pm - (Replying to @tdietterich and @twimlai @tdietterich) Would love to read you blog post on "understanding", but could not find a link. Can you link-a-link me?

11.9.19 6:20pm - I have hoped that the "Imitate RCT" approach will give way to the "interrogate nature" approach, in which RCT is just one way of interrogating nature, as in #Bookofwhy. The difference may seem academic; no so, when it comes to explanation, see

11.9.19 6:02pm - (Replying to @tanyacash21 and @gnshealthcare) I looked up GNS and found this: "Using Causal AI to answer  the Patient Response Question Biopharma companies have long struggled to identify the best responders for specific drugs." What's your method? Is it a secret? #Bookofwhy @smueller

11.9.19 5:30pm - (Replying to @olafhartig) Olaf, on the other hand, German streets are paved with 7,000 "Stolpersteine" (commemoratives stones), as I've learned from:, while American politicians' minds are paved with ignorance.

11.9.19 1:00pm - And I know reporters who are kind to kindness.

11.9.19 11:48pm - I was 2 years old on Kristallnacht, November 9-10 1938, It was a quiet night in Tel Aviv. My parents understood what's coming, but kept it from us, so we can grow up free. We did. And by the time we understood what kristallnacht meant, Egyptian war plains came attacking Tel Aviv.

11.9.19 2:29am - (Replying to @renatrigiorese) I'm not familiar with Khrennikov, and what I know about de Finetti comes from others. i.e., Diaconis & Skirms "Ten Great Ideas About Chance"declares de Finetti "the father of subjective Bayesianism"but I'm not convinced he could manage the "inevitable regret" problem. #Bookofwhy

11.8.19 3:37am - Here is another "must" in teaching philosophy of statistics: What logic gives us the authority to say something about an individual by observing other individuals, however similar? The question is raised here: and I wonder what philosophers say #Bookofwhy

11.8.19 3:37am - Here is another "must" in teaching philosophy of statistics: What logic gives us the authority to say something about an individual by observing other individuals, however similar? The question is raised here: and I wonder what philosophers say #Bookofwhy

11.8.19 3:10am - (Replying to @klts0 @EpiEllie and @HarvardChanSPH) Enticing title: "Un-conflating causal effects and intervention effects: saving the counterfactual baby while tossing the well-defined intervention bathwater." I hope Sharon un-conflates Rung-2 and Rung-3 of the Ladder. Else, re-conflation will triumph. #Bookofwhy

11.8.19 2:54am - (Replying to @IdoDaniel @GalitPeleg and @TheOnion) I am studying the faces of the delegates standing there, and I'm thinking: How do they manage to control their smiles? Super-humans! As the Mishna said: "Who is strong (Eizehu Gibor), he who can conquer his smile" (almost).

11.8.19 1:42am - (1/2) @bzaharatos , your Tweet reminded me how anxious I am to teach a course in Philosophy of Statistics (if they let me). Because there are so few philosophers who understand that the foundations of statistics lie in causality, not in statistics. I would start with Chapter 6,
11.8.19 1:42am - (2/2) because through paradoxes we see foundations. I would then go to the chapter on Rev Bayes, spiced with "why I am only half Bayesian?" and a few more goodies that I will try tweeting tomorrow, after some search. #Bookofwhy

11.7.19 11:57pm - (Replying to @tanyacash21) "On a bookshelf in a bar"? Can't think of a greater honor to a bar-avoiding author! (Nothing against the culture, but can't stand the noise -- do they have silent bars someplace?). But what do I see on Dylan's note: "grain of salt". Was he serious?

11.7.19 6:20pm - (Replying to @Yu_Ke__) Sorry, the expression "n order effect" is not known to me. Can we perhaps translate it to the language of SCM?

11.7.19 5:37am - (Replying to @Humblefool_14) I dont get it. Do you mean weird effects due to unanticipated eventualities?

11.7.19 4:13am - (Replying to @hirahira2835) No problem, write to my assistant BTW, what is "orange bon"?

11.7.19 4:11am - (Replying to @Humblefool_14) HMM. This section could stand expansion, I agree. But see if the discussion of Inverse Probability Weighting in Primer answers your questions Or, perhaps here: where mass movement is described in vivid colors? #Bookofwhy

11.7.19 4:03am - (Replying to @Humblefool_14) What is "second order effect"? Example please.

11.7.19 1:40am - I am trying to remain unemotional, but seeing these flags blend in solidarity bring tears to my eyes. Tears of common experience, common purpose, and of pride. Pride in a people that is able to give back what the world has given them - hope.

11.7.19 12:56am - (Replying to @Navatso_IR @RashidaTlaib and @RepAndyLevin) Of course not! It means only that their leaders cannot pose as seekers of a "peaceful solution", craving for an "honest dialogue". They should start with honesty: "We are bent on destroying you, but let's talk about the occupation".

11.6.19 11:53pm - (Replying to @Navatso_IR @RashidaTlaib and @RepAndyLevin) You're right. I've assumed that other readers are as tuned as I am to the fine nuances in the conflict. To clarify, for Palestinians, "peaceful solution" is "no Israel". For Israelis, it is "Coexistence: two states for two peoples, equally legitimate and equally indigenous."

11.6.19 10:30pm - (Replying to @RashidaTlaib and @RepAndyLevin) Those who insist on "an honest dialogue about the occupation" while refusing "an honest dialogue about co-existence" are incapable of leading an honest dialogue on any subject.

11.6.19 6:50pm - (Replying to @EinatWilf) I wish I could understand the neural architecture of a person who chose to head an organization committed to the perpetuation of human misery.

11.6.19 6:38pm - (Replying to @DKedmey) DAGophobia = Egophilia + Patriotism + Paranoia

11.6.19 4:16am - Thanks all for the pointers posted here to Turing-Bayes. As you can tell from #Bookofwhy, my mind only works with toy examples. I wish I could see a toy Enigma machine with just two wheels, showing the tradeoff that Turing faced between using Bayes rule and exhaustive search.

11.6.19 3:10am - I just watched Sharon McGayne fascinating talk on Bayes and I noticed that the scientific essence of Bayes rule is still not clear to the general public. Does anyone know of a simple description of how Turing used Bayes theorem to break the Enigma code?

11.6.19 1:30am - (Replying to @omaclaren @stephensenn and 4 others) I would hate to impose my own interpretation of "before". I will take whatever statisticians mean by it in their attempt to improve SE w/o causal vocabulary. My point is that even temporal information is insufficient. #Bookofwhy

11.6.19 1:10am - (Replying to @mendel_random and @stephensenn) "Causal thinking" was there all the time. In fact this thread revolves around the discovery that causal thinking drives even in purely predictive tasks. What #Bookofwhy claims is that there wasn't much beyond "thinking", because the "thinkers" lacked notation. Today we have it.

11.5.19 11:27pm - (Replying to @stephensenn @autoregress and 3 others) Great!!! #Bookofwhy commends David Cox (1958) for breaking statistical taboos and using a causal expression "affected by". Now, how about covariates measured BEFORE randomization. Are they safe, in purely predictive context? What do the textbooks say? Do you agree with them?

11.5.19 11:13pm - (Replying to @RonKenett and @stephensenn) I promise to keep a qualifier when its absence can be confounding. So far, your reference to aliasing was the first complaint I got since 1993. But our more pressing issue concerns purely predictive tasks which, presumably, should require no causal thinking. They do!! #Bookofwhy

11.5.19 11:00pm - (Replying to @stephensenn @autoregress and 3 others) I thought you will give us a statistical criterion to decide which variables should safely be controlled for, to reduce SE. Hundreds of statisticians on this education channel will sigh in relief when you agree that no such criterion exists. #Bookofwhy

11.5.19 10:49pm - (Replying to @RonKenett and @stephensenn) If statisticians have been using the word "confounding" in other meanings, different from Eq. (1), I will qualify my statement. But sticking to Eq. (1), I maintain: Confounding is not a statistical notion. And please note, I define things explicitly and unambiguously. #Bookofwhy

11.5.19 10:28pm - (Replying to @stephensenn @autoregress and 3 others) We are all concentrating on SE, nothing is missing. So far, all assertions about reducing SE (by variable control) had exceptions. To see it, please express your favorite (statistical) criterion for reducing SE, and I will produce a counter example for your enjoyment. #Bookofwhy

11.5.19 10:00pm - (Replying to @RonKenett and @stephensenn) In 1998, I wrote: "Why there is no statistical test for confounding, why many think there is, and why they are almost right" It was later incorporated into Chapter 6 of Causality. "Confounding" is defined as violation of Eq.(1), nothing more. #Bookofwhy

11.5.19 7:15pm - (Replying to @autoregress @AdanZBecerra1 and 3 others) In a regression context, at least, there are no "treatments" nor "effects", only "predictors" and "predicted". Can you restate your theorem in these terms.? #Bookofwhy

11.5.19 5:52pm - (1/3) This thread might come as a shock to regression analysts and perhaps to all statistically-trained followers of this education channel. One of the least disputed dictum in Statistic textbooks says that, in purely predictive tasks, to reduce standard errors, one should adjust
11.5.19 5:52pm - (2/3) for variables that are associated with the predicted outcome. Recent conversations on this thread drove me to refute this dictum and state that, even in purely predictive tasks, the language of statistics (eg. "associated with") is inadequate for deciding which variables
11.5.19 5:52pm - (3/3) should or should not be adjusted for. Causal models are needed for the task, which cannot be captured in the language of statistics. Counter-dictum examples abound. This means that to do statistics properly one should (occasionally) snap out of it. #Bookofwhy

11.5.19 2:44pm - (Replying to @stephensenn @AdanZBecerra1 and 2 others) Adjusting for variables that are associated with outcomes, because Stat textbooks say you should, CAN be a fundamental mistake. In general, using statistical vocabulary to decide what to adjust for should be a fail in Stat 1. It hasn't been, but it should be. #Bookofwhy

11.5.19 1:18pm - (Replying to @AdanZBecerra1 @fledglingStat and 2 others) These are good heuristics but insufficient. Adjusting for vars that are associated w outcome but not exposure may actually increase standard errors AND change causal effect. The rule is: Do not use statistical vocabulary when speaking "effect", or even "outcome", use #Bookofwhy

11.5.19 1:04pm - (Replying to @stephensenn @AdanZBecerra1 and 2 others) This is what R. Fisher said, when asked about mediation, and fumbled with ANOVA. See "Identifying a mediator" is easy, using our mind DAG, but how to handle it in assessing mediation is not easy, as Fisher's fumbling shows us. #Bookofwhy

11.5.19 6:58am - (Replying to @fledglingStat @robertstats and @stephensenn) I do not believe any trick will enable you to distinguish good from bad "explainer" or "accounter for" of "partitioner" using associations vocabulary. But I'll keep my mind open. #Bookofwhy

11.5.19 6:39am - (Replying to @fledglingStat @robertstats and @stephensenn) It does indeed. I was talking about the confounding step. Speaking about "explain outcome variability" is interesting, because "explain", again, is a causal, not statistical notion. I am curious how statisticians defined it in terms of "associations" #Bookofwhy

11.5.19 6:27am - (Replying to @stephensenn) I have assumed we are both in the context of estimating causal effects, where covariates enter into the denominator of PS because they are presumed to reduce bias (explained in When chosen to reduce variance, statistical considerations set in. #Bookofwhy

11.5.19 5:53am - (Replying to @stephensenn) Choosing covariates by association is fundamentally wrong, regardless of what associations are considered. Covariates should be chosen to reduce confounding, and confounding is a causal, not statistical concept. #Bookofwhy

11.5.19 4:25am - (Replying to @yudapearl @firstmn59 and @Cahliveira) The tyranny has little to do with DAGs (though those have been tabooed by leadership since 1995) It has more to do with the paradigm imposed by "experimentalists" which forbids their faithfuls from seeking TPT (Transparency, Power, Testability). #Bookofwhy

11.5.19 4:11am - (Replying to @firstmn59 and @Cahliveira) The tyranny comes not from econs., but from their leadership. Try sending an email to the editor of any "top" journal and ask whether he/she can refer you to ANY paper (or review, or survey) describing methods of mediation analysis. Please share their answers with us. #Bookofwhy

11.5.19 2:40am - I thought it was a dream: "demanding Iraqi Jews to come back "home"'. I rushed to show it to my wife, who was expelled from Iraq in 1950, to Israel. Her answer: These people will never understand what "home" is; we never felt "at home" in Iraq, though we stayed there 2,536 yrs.

11.5.19 1:42am - Great idea!! I am honored to be one of the first to speak at this forum, and I hope it becomes an educational tool to the entire data-science community, with lectures from variety of disciplines teaching what can be done with TPT (Transparency, Power and Testability). #Bookofwhy

11.5.19 1:30am - (Replying to @ArielElyseGold) "deaths of people"? Unheard of!! Champions of humanity and protectors of human rights would never dream "calling for the deaths of people." If we only psych ourselves into a self-hating Stockholm Syndrome, we'll see things as they really are, not as we wish them to be.

11.4.19 11:34pm - (Replying to @yudapearl and @kareem_carr) Apropos Psychometric Society, I met two past presidents. The first, the late Rod McDonald, wrote to me (2009): "The best way to discuss moderation or mediation is to set aside the entire literature on these topics and start from scratch". The 2nd did not follow Rod's wisdom.

11.4.19 10:50pm - (Replying to @kareem_carr) Students tend to underestimate the clout they have. If I were President of the Psychometric Society and a student wrote to me how he saw the name of my organization tarnished in a serious Tweeter conversation, I would jump 10 ft and become an expert mediation analyst. #Bookofwhy

11.4.19 4:25pm - Action should overcome depression. How about writing to the leadership of psychometric society (exists?) whether they are aware of the reputation of their literature. I bet they are waiting for an excuse to reinvigorate the editorial staff of their leading journals with new blood

11.4.19 2:33pm - This is an amazing finding: 97% studies still in the stone age! Is Psychometrics under same tyranny as Econometrics? Here's a great PhD topic (ech. Kuhn):"The psychology of scientific revolutions". Is it institutionally imposed or intellectually chosen? Any historians? #Bookofwhy

11.4.19 6:38pm - (Replying to @glarange72) I never practiced "meditation" but, comes to think about it, it does take some meditation to explain what happened to economic education in the past few decades. Bemoaned here:, but echoed nowhere enough to shake "top" econ. journals. #Bookofwhy

11.4.19 6:38am - (Replying to @glarange72) I never practiced "meditation" but, comes to think about it, it does take some meditation to explain what happened to economic education in the past few decades. Bemoaned here:, but echoed nowhere enough to shake "top" econ. journals. #Bookofwhy

11.4.19 6:17am - Why I said "especially economists!"? Because there are many enlightened economists among our readers, educated on "mostly harmless", who cannot wait for "top" econometric journals to open their gates to modern causal analysis. And economics is all about mediation #Bookofwhy

11.4.19 5:03am - Readers (esp. economists!) interested in a gentle introduction to modern mediation analysis will find this post useful: It contrasts modern analysis with the Baron & Kelley tradition, gives examples, and explains the meaning of Natural Effects. #Bookofwhy

11.4.19 4:41am - Sharing a space-photo of Israel by lady astronaut (and Howard professor) Jessica Meir, who remembered her father's first country after escaping from Iraq. We can clearly see the Red Sea, the Sea of Galilee, and the Suez Canal. My biblical home town is there too, under the clouds.

11.4.19 2:51am - (Replying to @dmi3k) Agree, the article would have been better written with one or two DAGs, to show the competing theories, and what each tells us about predictions and interventions.But it is still an interesting article and a nice summary of this line of psychological studies. #Bookofwhy

11.4.19 2:29am - (Replying to @dmi3k) Causality is confusing? God forbid! It's the simplest scientific concept we have. It was made confusing by statisticians who thought that anything that is not in the data must be either "confusing" or "super-natural". I hope they think differently today (do they?) #Bookofwhy

11.4.19 2:04am - (Replying to @DataDonors @jan_ni and @welchering) My German speaking neighbors, in the town where I grew up use to say: "stimme voll und ganz zu" which, to me, sounded like: "Dont you dare say it again!" I remember them fondly. #Bookofwhy

11.3.19 4:58am - (Replying to @RDMetcalfe @jhaushofer and @PHuenermund) I might agree with you, once we explicate precisely what "all else equal" means. But I cannot agree on "because you better understand". "Understanding" was yesterday's excuse for inaction; today we can formalize and quantify whatever needs understanding. #Bookofwhy

11.3.19 4:28am - (Replying to @jhaushofer and @PHuenermund) In this debate: I stressed: " Observational studies have the virtue of observing large number of people in their own natural habitat instead of an artificial experimental setting, marred by selection bias" #Bookofwhy

11.2.19 8:59pm - I used to think that bootstrapping is a statistical estimation technique, of little use in causal inference. This paper, however, tells about causal bootstrapping It needs to be listened to. #Bookofwhy

11.2.19 7:56pm - It is well known (is it??) that covariate-adjusted and propensity-score estimators are asymptotically equivalent (shown eg. here ). This recent paper combines the two to improve small-sample performance. #Bookofwhy

11.2.19 7:15pm - A new article crossing my desk provides a procedure, new in SAS/STAT® 15.1, for analyzing graphical causal models. I hope readers find it useful #Bookofwhy

11.2.19 3:48pm - True, my article deals only with the moral/historical significance of the Balfour Declaration. For the demographic/political forces behind the scene I recommend Kramer's eye-opening article

11.2.19 3:06pm - (Replying to @rkarmani) I do not know of any, sorry. But my ignorance does not say a thing about existence. #Bookofwhy

11.2.19 6:04am - (Replying to @JustinSandefur) And I thought (naively) that I convinced Deaton here: that some of his critics were outdated. It's hard to convince an economist, but his paper is fun to read, especially for its survey of economists' thoughts #Bookofwhy

11.2.19 5:46am - (Replying to @Abel_TorresM @dmonett and 2 others) As I am sure you know, temporal information is not sufficient for eliminating confounding, as the debate about smoking--->cancer reminds us. We need scientific knowledge about who is listening to whom, eg a DAG. #Bookofwhy

11.2.19 12:18am - Your history professor understood the music of history. I wouldn't be alive if it weren't for the Balfour Declaration (1917), which gave my grandfather the push to leave boiling Europe , rebuild a Biblical town (1924) and then create a miracle called Israel (1948). Kudos!Balfour!

11.1.19 11:30pm - (Replying to @Abel_TorresM @dmonett and 2 others) I do not think it is correct without explicating what we mean by "based on correlation". If it means "based ONLY on correlation" I would say: No. If it means "based on correlation and causal assumptions" I would say: Sure, this is what #Bookofwhy is all about.

11.1.19 10:32pm - My,My! It's November 2nd! And I almost forgot to invite you to celebrate with me the Balfour Declaration: The first international recognition of a people's right to a homeland (1917). I even wrote an article on its universal significance: One of my best!

11.1.19 5:43am - (Replying to @GuillermoBurr) I did not realize that this is what the "evidence-based" movement was after all these years. Why did'nt they tell us? They should have called their aspiration "forces-based" movement, because "evidence" usually means "an observed piece of data".#Bookofwhy

11.1.19 5:03am - We know that institutions can turn creepy, its in their nature. What I cannot understand is how the delegates, fellow human beings, can face their families, back home, when their children ask: "And you Papa were part of this Zoo?"

10.31.19 9:00pm - Discovered an interview I gave in Oct 2018, with a punchline saying: "Knowledge lies not in dry facts but in the forces BEHIND the facts." Sounds obvious? Not to folks who think big data means big knowledge. #Bookofwhy

10.31.19 12:24am - (Replying to @ClaudeAGarcia) I was not aware of this study, thanks. My paper asks what does it take for the "illusion of understanding" to be less of an illusion and more grounded in science, data and logic to earn the title "rational understanding" #Bookofwhy

10.30.19 8:42pm - Sharing the final version of a chapter for The Handbook of Rationality It now seems strange to me that philosophers have spent so much effort on rationality of preferences and so little on rationality of envisioned consequences. #Bookofwhy

10.30.19 5:10pm - To readers in Brazil, and to followers everywhere, a bit of fresh air and good will from Haifa, where people can still enjoy a fun game of soccer, spiteful of dark clouds and hopeful of (multiple) better worlds.

10.30.19 2:30pm - (Replying to @David_desJ and @anthonysamir) My argument is based on specific evidence, as well as long experience in spotting Zionophobes and reading their modus operandi. The "never fails" is not a premise but a fact among Omar's constituency, in case you have watched them in action.

10.30.19 2:12pm - (Replying to @David_desJ and @anthonysamir) Beg to disagree. "Islamic" is too heterogenous an entity to serve as a focus of obsession, varying wildly from Erdogan, to Ayatola, to Sisi and Saudia. Zionophobia is simple, focused, emotionally charged, and never fails to arouse the worst in people.

10.30.19 4:17am - (Replying to @DrMikeH49 and @ErakatSaeb) And as long as no school teacher in Ramallah mentions this "commitment".

10.30.19 4:05am - What a happy day! When I was a boy, 6-7, I used to visit my aunt who lived in King Cyrus street (Melech Koresh) in Tel-Aviv. When I asked my teachers who King Cyrus was, they asked my to open my Bible, book of Ezra, it was all there!!! Happy King Cyrus day!

10.30.19 3:44am - (Replying to @anthonysamir and @David_desJ) Not "selective attention" but "selective obsession". She cannot offend Sultan Erdogan as long as he supports Hamas.

10.30.19 2:18am - I actually called PS "ingenious". The only thing I "demolished" was the myth that it has anything to do with (asymptotic) "bias reduction". It is a powerful estimator, once you make sure that the covariates satisfy the back-door criterion. See #Bookofwhy

10.30.19 1:34am - (1/ ) (Replying to @TatThangVo1 @LauraBBalzer and 9 others) I commend you for taking the lead and explaining to statisticians how they should think and do meta-analysis in this century. You should follow it with a glossary of terms to help them and us communicate across jargon barriers. I hope they resonate favorably to your heresy.I'm
10.30.19 1:49am - (2/2) (Replying to @yudapearl @TatThangVo1 and 10 others) wondering though whether you think traditional meta-analysts (say S Normand) would feel more comfortable thinking "ignorability" or thinking "graph separation"? Traditionalists shun both, to their detriment, but the latter is more human. Awaiting to see feedback. #Bookofwhy
10.30.19 3:10am - (3/ ) (Replying to @yudapearl @TatThangVo1 and 10 others) Apropos, here is Wiki's definition: Meta-analysis is the statistical procedure for combining data from multiple studies. When the treatment effect (or effect size) is consistent from one study to the next, meta-analysis can be used to identify this common effect.
10.30.19 3:19am - (4/ ) (Replying to @yudapearl @TatThangVo1 and 10 others) When the effect varies from one study to the next, meta-analysis may be used to identify the reason for the variation." Someone should correct it. (1) The reason CANNOT be identified by ANY statistical means. (2) Today we can do more than just "identify reasons". #Bookofwhy

10.30.19 1:23am - (Replying to @thebyrdlab) The trouble with the "ignorability" assumption is not that those who make it do not know how to test it (or even if it can be tested) but that it is so far removed from what they know about the problem, that no one can argue for its plausibility or implausibility.#Bookofwhy

10.29.19 9:14pm - This is a wonderful paper that should be read by every ML researcher concerned with explainability. I dont know how it escape my attention. I would shorten it more, skip the "according to so and so.." and present ML folks with a computer-minded taxonomy of explanation #Bookofwhy

10.29.19 1:47pm - Those who deem "ignorability" crucial are escaping the light of day. Which explains why they deem it "crucial". #Bookofwhy

10.29.19 5:54am - (Replying to @EpidByDesign @LauraBBalzer and 11 others) Your abstract says: "we show that even perfect internal validity does not ensure that a causal effect will be unbiased in a specific target population." Is this surprising? Why would IV have anything to do with EV? #Bookofwhy

10.29.19 4:39am - (Replying to @FriedrichHayek) I have that hunch too, but I need help in translating their ideas from verbose philosophical writing to computer age: What information do we need, and how to represent it, so as to generate rational explanations for actions intended as well as actions already taken. #Bookofwhy

10.29.19 1:11am - I have a strong hunch that ML discussions on explainability could benefit from the centuries of "philosophy of action", for example Likewise, philosophers should benefit from our new ability to represent "actions" formally in a machine. #Bookofwhy

10.29.19 12:56am - (Replying to @andrewthesmart @jainfamilyinst and 19 others) To be more specific, once we assess the causal connections among variables in a given population, we can also assess "the degree of discrimination" exhibited by that population. The latter is DEFINED in terms of (not "defined by") the former. #Bookofwhy

10.29.19 12:46am - (Replying to @matsouaka @LauraBBalzer and 9 others) Trying to advise readers who have spent many hours studying classical transportability (eg Would they be able to solve new problems after spending many more hours on Issa et al ? Or is it the same set of problems? Perhaps weaker assumptions? #Bookofwhy

10.28.19 6:16pm - (1/ ) My screen got blessed by an intriguing Open Access volume titled: Free Will, Causality, and Neuroscience. Chapter 3 reads: When Do Robots Have Free Will? A question we address in #Bookofwhy. Unforturtunately,
10.28.19 6:16pm - (2/2) I got spoiled by Twitter and lost the ability to go over pages +pages of introductions. Can anyone extract from it the meat: When would we say that a robot passed the Turing test for Free Will? #Bookofwhy

10.28.19 5:50pm - (Replying to @jwhogan42 @LauraBBalzer and 9 others) Naive question: Anyone knows what these two papers add to the work on Data Fusion described here: The abstracts read the same, and the texts do not offer comparisons. Any help? #Bookofwhy

10.28.19 3:52pm - (Replying to @andrewthesmart @jainfamilyinst and 19 others) DAGs postulate individual-level relations. But data only allows us to estimate population-level quantities. BTW, "discrimination" is not a variable, but a property of behavior, the degree of which can be estimated from data. #Bookofwhy

10.28.19 3:36pm - (Replying to @LauraBBalzer @eliasbareinboim and 8 others) What's the occasion? Is NIH funding a new initiative? (They should!). My favorite is this one:, primarily because it raises so many questions that have not been answered, all on a tiny example: X-->Z-->Y. #Bookofwhy

10.27.19 8:33pm - (Replying to @MiriamElman @TheAENetwork and @UMassAmherst) He needs support from more than one organization. Most importantly, UMass students and faculty should express their commitment to inclusion and a hate-free campus. Other Chancellors have condemned hateful meetings before, yet no one has dared do it to the smoke-covered BDS

10.27.19 3:20pm - A moment of silence for the eleven Tree of Life Victims. A personal connection: Joyce Fienberg was the wife of Professor Steve Fienberg of CMU, a pioneer in categorical data analysis, a renaissance man and a true friend. #Bookofwhy

10.27.19 5:29am - (Replying to @s_jagabathula @fuiud and @PHuenermund) I've seen this paper before, but thanks for reposting - I should read it again. Ad-placement is indeed an industry that pays many ML salaries and is loaded with not-trivial causal and counterfactual considerations, see eg., Needs rereading. #Bookofwhy

10.27.19 12:44am - Something wonderful has happened to me on Twitter. I fell in love with my own slogan: "Transparency, Power and Testability" (TPT, defined below). I am proposing it as a litmus test, to distinguish genuine CI from its many claimants. What do our TPT followers think? #Bookofwhy

10.26.19 8:44pm - (1/ ) Readers keep asking: "What's so special about the Chancellor of UMass?" Examime is his original statement: . As you can see, he does not stop at the mantra: "Academic boycotts are antithetical to academic freedom" but continues with a moral indictment:
10.26.19 8:44pm - (2/ ) "the BDS position in general fails to acknowledge the humanity on the Israeli side of the conflict" and "is the antithesis of our commitment to inclusion". Finally, a Chancellor who speaks like a Chancellor, and fearlessly tells the campus where BDS stands on the moral scale

10.26.19 7:55pm - (1/ ) I am retweeting, because the question: "Compare the PO vs. DAG approaches on applied, 'real-life' data" comes up once a week. Here it is: Take any applied paper labeled "PO" and re-label it "DAG" after adding to it three tiny ingredients: (1) Show how the modeling assumptions
10.26.19 7:55pm - (2/ ) emerge organically from our scientific understanding of the problem, (2) Show if there is anything we can do if it doesn't thus emerge, and (3) test whether those assumptions are compatible with the available data. Once added, Bingo! We have an "applied DAG" approach,
10.26.19 7:55pm - (3/3) demonstrated on "real life" data, that can be compared to the "applied PO approach" or "applied quasi-experimental approach" on all counts, especially on: (1) transparency, (2) Power, and (3) Testability. #Bookofwhy

10.26.19 7:15pm - (Replying to @SamuelW09371956 @fuiud and 2 others) Why not? Epidemiologists are so much better than me in conducting experiments that I chose not to compete. Economists are better too, but their mind has been locked into the "credibility" movement, and only now their students are saying: "Wow! No one told us" #Bookofwhy

10.26.19 5:37pm - (Replying to @fuiud and @PHuenermund) Most of applied work in epidemiology since 2000 uses DAGs. So much so, that 2018 saw a couple of papers protesting "the tyranny of DAGs in epi". But speaking of high school, @smueller has been teaching HS using Primer. Kids understand that toy problems are"applied work"#Bookofwhy

10.26.19 5:20pm - (Replying to @fuiud and @PHuenermund) When I said "people who never used DAGs" I meant: "people who never used DAGs, even on toy problems, because toy problems force them to deal with issues that "applied works" allow them to hide. #Bookofwhy

10.26.19 5:54pm - (Replying to @fuiud @SamuelW09371956 and 2 others) When I was in high school I fell asleep reading other people's experimental work, all I wanted to know was how I can apply the method learned to my own problems, or to problems I am likely to encounter in the future. #Bookofwhy

10.26.19 4:45pm - (Replying to @fuiud and @PHuenermund) Your description of "DAG approach" comes from people who never used DAGs. Correct description: Take any applied PO+IV study, and add a tiny question: How do we know, given our problem description, that Z is a good IV? Once you answer it, you got an "applied DAG" study. #Bookofwhy

10.26.19 4:30pm - (Replying to @fuiud and @PHuenermund) If it is seeing PO vs DAG, then the answer is even simpler: Take any applied paper labeled "PO" and re-label it "DAG" after adding to it one tiny ingredient: showing how the assumption W||X,Y(0),Y(1) emerges from students understanding of the problem. Bingo! PO vs. DAG #Bookofwhy

10.26.19 4:16pm - (Replying to @fuiud and @PHuenermund) If it is seeing PO vs DAG, then the answer is even simpler: Take any applied paper labeled "PO" and re-label it "DAG" after adding to it one tiny ingredient: showing how the assumption W||X,Y(0),Y(1) emerges from students understanding of the problem. Bingo! PO vs. DAG #Bookofwhy

10.26.19 4:03pm - (Replying to @fuiud and @PHuenermund) If it is seeing PO vs DAG, then the answer is even simpler: Take any applied paper labeled "PO" and re-label it "DAG" after adding to it one tiny ingredient: showing how the assumption W||X,Y(0),Y(1) emerges from students understanding of the problem. Bingo! PO vs. DAG #Bookofwhy

10.26.19 6:41am - (Replying to @RogerFrigola and @DOAJplus) No, I do not know. In what context can it be useful?

10.26.19 1:58am - (Replying to @fuiud) Did you look into the example I suggested, p.67 of: I have found it to be excellent for teaching, because it demonstrates both backdoor and frontdoor and it starts with data (Table 3.1), not a distribution. Try it. #Bookofwhy

10.26.19 1:27am - (1/ ) (Replying to @gerdienja @analisereal and 2 others) Very interesting paper. I never heard of Guala theory, stating: "if one has (a) some controlled initial conditions in the laboratory and (b) some observed experimental result on the one hand and given (c) some observed properties of the target system and (d) some observational
10.26.19 1:34am - (2/ ) (eplying to @yudapearl @gerdienja and 3 others) data on the other, then (by analogy) c stands in the same (causal) relation to d as a stands to b (Guala 1998)." Did you check if this rule works on the 3 canonical examples in Fig. 1 of ?? Off hand it seems it does not. Are you in the mood to improve
10.26.19 1:47am - (3/3) (Replying to @yudapearl @gerdienja and 3 other) Guala's theory so that it covers what we have learned about extrapolation in the past decade? A summary of what we can do today is given in my comments on Deaton & Cartwright which you cite. I think Guala would appreciate it. #Bookofwhy Cor Fig1->Fig3

10.25.19 7:50pm - Anders, I failed to understand your post. You start by describing the problem nicely but, then, instead of specifying what information you have or don't have, you rush to tell readers what's wrong with Bareinboim and Pearl. Completing the specification would help #Bookofwhy #Bookofwhy

10.25.19 7:47pm - (Replying to @AndersHuitfeldt and @analisereala) I was surprised to find out that parametric assumptions were NOT crucial for so many problems. So I am curious to see one problem, fully specified, where it IS crucial. #Bookofwhy

10.25.19 5:02pm - (Replying to @AndersHuitfeldt and @analisereal) I am honored to be designated as the one who changes the way "almost everyone" is talking. If we know Pr(Y=y|do(x)) for every x and y, surely we can compute ratios, differences, logarithms and arctangents of various combinations of x and y. #Bookofwhy

10.25.19 3:38pm - (Replying to @AndersHuitfeldt and @analisereal) Equating "effect sizes" with "parametric" analysis is new to me. Surely, "size" stands for a number. But that does not mean that we should commit to a specific functional form in analyzing questions of identifiability or heterogeneity. #Bookofwhy

10.25.19 3:31pm - (Replying to @AndersHuitfeldt and @analisereal) There are 2 reasons for doing "nonparametric" analysis: 1. Lacking knowledge of param. form. 2. Generality. If I can write an effect nonparametrically, eg. ACE=P(y|x), then this should hold for ANY parametric form in which I choose to estimate P(y|x), depending on data.#Bookofwhy

10.25.19 3:20pm - (Replying to @AndersHuitfeldt and @analisereal) I am not interested in winning friends as much as I am interested in doing things correctly for the set of problems that I find challenging. Please tell us what definition "makes more sense", why hide it? #Bookofwhy

10.25.19 3:15pm - (Replying to @AndersHuitfeldt and @analisereal) I believe my def. of "heterogeneity" is the same as yours. Here it is: "Treatment effect varies across populations". Why mystify it? Traditional Meta-analysis deals with "distributional variations across populations" which is a different species. #Bookofwhy

10.25.19 1:26pm - I agree with @analisereal observation that a complete solution to external validity problems is available for the nonparametric case. Moreover, the many "meta-analytical" papers that I have read, are simply not dealing with the problem of heterogeneity. #Bookofwhy

10.25.19 1:16pm - (Replying to @fuiud @PHuenermund and @Jabaluck) Every applied paper is using DAG for identification, some explicate portions of it, some explicate conclusions based on it (eg ignorability) and most try to keep the DAG secret, informal. A DAG is simply the way scientific knowledge is stored in the mind, no escape. #Bookofwhy

10.25.19 5:07am - (Replying to @PHuenermund) But what do you make of the fact that so many people "liked" to believe that ***convolution solves causality**. Are they really unaware of the "curve fitting barrier" or just expressing a wish that what they understand would solve what they don't ? Puzzled. #Bookofwhy

10.25.19 3:19am - I think I heard you from outside, and I have answered it inside at great length. No, DAGs do not replace that crucial step, they do much more. They give you a language to demystify that mental step called "research design" by bringing it to where knowledge resides. #Bookofwhy

10.25.19 1:28am - Dear colleagues at UMass, Amherst, Please convey to your Chancellor, Kumble Subbaswamy, my deepest thanks for demonstrating exemplary leadership, and condemning the BDS movement on hard moral grounds, the only effective way of exposing racism on campus.

10.25.19 12:30am - For readers asking for hints in solving the puzzle in this slide:, the setting assumes that Z2 and W2 are unmeasurable, while other variables are. We need to identify the causal effect of Z3 on Y (which is confounded) and we ask how to do it using an IV?

10.25.19 12:13am - (Replying to @boredyannlecun and @ylecun) My twitter receives many: "XYZ solves causality", "We have been doing causality since 1988" etc. I cannot respond to such statements without seeing how XYZ breaks through the "curve-fitting barrier" of a simple toy problem, such as the many examples described in #Bookofwhy.

10.24.19 10:15pm - From theory to algorithms. I call on colleagues, members of the National Academy of Science, National Academy of Engineering and National Academy of Medicine: Let's jolt these 3 organizations to take action on a matter that is central to their charters and vital to our community?

10.24.19 9:28pm - One promising idea: The creation of a non-profit organization that, leveraging modern technology, would compete with giants like Elsevier by promoting free dissemination of scientific research, as opposed to impeding access to such information. NSF,NIH and NAS should initiate it.

10.24.19 6:00pm - An opinion piece that should worry us all: An overhaul is needed to make science publishing fair and open. This, I believe, should be top priority for NSF, NIH, NAS, even Darpa, ie., all Gov's agencies in charge of scientific research. #Bookofwhy

10.24.19 5:23am - (Replying to @DaveBrady72) Not sure what kind of "post-treatment control" you find hard to teach. Does it appear in any of the models discussed in "crash course in good and bad controls"? linked here:

10.24.19 3:41am - I had great fun dancing with students over the hurdles of causal inference from A to Z. The slide I have enjoyed most was this:, since most students were trained in "mostly harmless", and had to be updated with practical tools. Solution anyone? #Bookofwhy

10.24.19 4:23am - @Susan_Athey Sylvia turned out a brilliant problem solver. She also taught me that the word "inference" has different meaning to different people. This is what I tried to do throughout the lecture, to define 'inference". Enjoyed it thoroughly. #Bookofwhy

10.23.19 12:08am - (Replying to @stephensenn and @kaz_yos) Not necessarily. "What if" asks for "effects of causes", whereas "Why" asks for "causes of effects". See how beautifully the two are handled in counterfactual logic: #Bookofwhy

10.22.19 6:02am - (Replying to @jazchaz) Thank you Raelle, and may the Muse of History acknowledge the music of Charlie and Jasper. I will be singing with them. Today is Shmini Atseret, the day Danny was Bar-Mitzva-ed at the Western Wall, in 1976, and joined the soul of our people.

10.22.19 4:35am - Following my talk at the Business School, I will be heading to Bing Concert Hall for the Daniel Pearl World Music Days concert at 7pm,, and sing for world sanity.

10.22.19 4:31am - Colleagues at Stanford and neighborhood may be interested in a talk I will be giving tomorrow, Oct. 23, 4:30pm at the Grad School of Business, "Class of 1968" bldg, Room 106 Topic: Econometrics and Causal Inference

10.21.19 8:45pm - Beyond love and hate, the innocent and curious asks: "Do they cross the 'curve fitting' barrier?" #Bookofwhy

10.21.19 5:45pm - (Replying to @ShMMor @thomaschattwill and 5 others) Batya should not have left, and her article should have informed the public that Forward Magazine will be joining others in making Zionophobia the ugliest word in town, because it IS and because the harassment she experienced will continue till it IS. See

10.21.19 5:09pm - (Replying to @alanmpardo) Sounds like an inspirational battle cry: "Let the reading commence!". I hope it leads to an easy victory. After all, the enemies are exhausted, and history is on your side! Charge! #Bookofwhy Batya should not have left, and her article should have informed the public that Forward Magazine will be joining others in making Zionophobia the ugliest word in town, because it IS and because the harassment she experienced will continue till it IS. See

10.21.19 4:13am - (Replying to @JaredJacobsen2) Thanks for re-tweeting this beautiful explanation of the distinction between rung-2 vs. rung-3 causation. This question comes back again and again, primarily because PO-researchers can't get it (cultural blinders), and they still rule the planet. Or do they? #Bookofwhy

10.21.19 3:50am - (Replying to @patrick_s_smart @KateRaworth) . Curious to know what Doughnt Econ is? If you can summarize it in one Tweet, I promise to do one for you on #Bookofwhy

10.20.19 11:50pm - Todah Rabah Maestro Zubin Mehta. Every October, since 2002, you have dedicated one of your concerts to the Daniel Pearl World Music Days. Today, as you lead the Israeli Philharmonic Orchestra for the last time, we salute your vision, artistry and humanity. To Life!

10.19.19 12:01pm - While some media outlets are interpreting the recent Nobel announcement as a rebuke of those who challenge RCT hegemony, this is not the dominant view among economists. This article takes a more balanced view of RCT economics #Bookofwhy

10.18.19 10:54pm - (1/ ) Regression analysts, and this means tens of thousands of smart statisticians, are recovering from a century of internal anguish, in which they had to think causes and do regression. They now rejoice the power of writing and speaking the same language -- causal graphs, even
10.18.19 10:54pm - (2/2) when analyzing pure prediction tasks. I would encourage the authors, but make sure the word "causal" is not added to the title just for riding the causal hype. We need to secure the reputation of "causal inference" as a new way of viewing scientific problems. #Bookofwhy

10.18.19 2:02am - My fingers are frozen, would I be able to Retweet the Untweetable?

10.17.19 10:07pm - We all lament the herd-like mentality that social media is breeding. But how can we account for the mentality of sovereign states, members of the United Nations, an organization ordained to be the moral guardian of the human race??

10.17.19 5:58pm - (Replying to @gfrison) Great "seeing" and "doing" experience. One comment: the term "Causal Bayesian Network" was defined (Causality, ch. 1) by interventions, not counterfactuals. For counterfactuals (Rung-3) we need functions eg. y=f(x,z,eps), named Structural Causal Models (SCM). #Bookofwhy

10.17.19 12:17pm - (Replying to @rwolffoot @_julesh_ and 3 others) You are right. The firing-squad example in ch. 1 of #Bookofwhy is deterministic and computing whether "The prisoner would still be dead had soldier A not fired" requires non-standard logic.

10.16.19 11:22pm - (Replying to @tvladeck @f2harrell and 14 others) Taking a Bayesian viewpoint (which I do not recommend), the unique problem of CI is to compare DAGs that have same likelihoods yet produce opposite answers. #bookofwhy

10.16.19 11:12pm - (Replying to @kareem_carr and @SpenceKjell) Why? Do you know anyone who has not read it yet? #Bookofwhy

10.16.19 11:10pm - (Replying to @f2harrell @tvladeck and 14 others) @f2harrell , Sticking to my "input-output" religion, can you tell us in "input-output" terms how this paper proposes to answer causal questions about a population? #Bookofwhy

10.16.19 10:59pm - (Replying to @FJnyc and @IzaTabaro) Mehdi Hasan and other eloquent racists will continue their hate-song with impunity until our leaders and writers start shaming them with the one ugly label they cannot dare to disown: "Zionophobe".

10.16.19 10:43pm - (Replying to @Humblefool_14) I do not see anything wrong with this article, every sections I start sounds reasonable. But I believe that in order to formulate "general principles of explicability" we need first to solve a toy explanation, e.g., "what caused the fire?" #Bookofwhy

10.16.19 10:27pm - (Replying to @austinvhuang @zacharylipton and @ipam_ucla) I see it for the first time, and I can't resonate to it because I can't see the input-output relationship. That is, what kind of questions we will be answer with this new system, and what do we need to specify to get it running. Does anyone see it? #Bookofwhy

10.16.19 5:30am - (Replying to @f2harrell @stephensenn and 13 others) Fair enough. All graph-based CI tasks take such uncertainty into account, since none relies on the assumption that "treatment improves" things. Improvements, if any, are inferred from {data+qualitative assumptions}, not assumed. #Bookofwhy

10.16.19 2:53am - Media hype aside, this paper by Duflo etal is a beautiful exposition of the philosophy of experimental economics in 2007, when economists were isolated from new developments in causal research. Of primary interest is Section 8, External Validity.#Bookofwhy

10.16.19 2:21am - Strange mediatake: In backdrop of Economics Nobel announcement, (an incriminating) letter panning RCT surfaces Business Standard Among the signatories: "Judea Pearl of Columbia University" Pleading innocent on all counts #Bookofwhy

10.15.19 8:32pm - (Replying to @AdanZBecerra1 @boback and 15 others) For a smoker, refraining is a giant treatment

10.15.19 8:26pm - (Replying to @boback @Lester_Domes and 14 others) I believe the 1964 decision to proclaim smoking a major cause of lung cancer is an example; it was supported by plausibility arguments and observational studies. #Bookofwhy

10.15.19 8:17pm - (Replying to @boback @Lester_Domes and 14 others) So far, I've learned that EBM is whatever physicians do, plus preference to published RCT's studies. Anything I've missed in my attempt to understand why so many people swear by EBM ?#Bookofwhy

10.15.19 8:10pm - (Replying to @boback @dailyzad and 14 others) That's a nasty thing to say.

10.15.19 5:52pm - (Replying to @dailyzad @boback and 14 others) Twitter is great for transmitting (1) substance or (2) poetry. Confusing when used for transmitting broad "sounds like" or "seems like" #Bookofwhy

10.15.19 4:56pm - (Replying to @Lester_Domes @stephensenn and 13 others) So, what does "quantification" tell us that is not told by causal calculus? Is it statistical properties of specific estimators on finite samples? or identification opportunities under specific parametrization of the model? #Bookofwhy

10.15.19 4:48pm - Reading an interesting paper on algorithmic fairness: "Unfairness" = The presence of undesirable or impermissble path-specific effects of sensitive attributes on outcomes. #Bookofwhy

10.15.19 2:53pm - (Replying to @f2harrell @stephensenn and 13 others) "general quantification weakness" seems too general and cryptic to me. Specific weaknesses will be addressed with specific attention or, better yet, with specific solutions #Bookofwhy.

10.15.19 2:42pm - (Replying to @Lester_Domes @RonKenett and 14 others) "And the sailors said, come and let us cast lots, to find out who is to blame for this ordeal." (Jonah 1:7)

10.15.19 2:15pm - (Replying to @goodmanmetrics @JadePinkSameera and 13 others) I have to take your question seriously because our Twitter conversations are loaded with complaints such as "Isn't it just traditional statistics" or "Isn't it textbook economics?" and the complainers are prominent researchers who need guidance and respect. #Bookofwhy

10.15.19 9:35am - (Replying to @goodmanmetrics @JadePinkSameera and 13 others) This is easy. The whole #Bookofwhy is dedicated to this question and shows in 10 chapters what questions we can answer through "causal inference" that we could not answer by traditional statistics. I'll be glad to answer any specific question about the book, as I have on Twitter.

10.15.19 9:07am - (Replying to @goodmanmetrics @stephensenn and 13 others) You deserve a Nobel for taking my question seriously and, if I may continue, does the EBM enterprise come with principles of distinguishing valid from anecdotal evidence? If yes, are those principles a subset or a superset of those analyzed in the "causal inference" literature?

10.15.19 7:08am - (Replying to @JadePinkSameera @ShannonBrownlee and 13 others) I would give a Nobel Prize in Education (if it existed) to any person who can explain to ordinary folks (eg me) what "Evidence Based Medicine" is, and how it differs from traditional medicine. #Bookofwhy

10.15.19 5:21am - (Replying to @AndersHuitfeldt and @PHuenermund) We are discussing a level of understanding that allows researchers to judge assumptions HARSHLY. Do you honestly believe that any mortal can judge the plausibility of "as if nature conducted a RCT...." in even simple situations?Say those discussed here:

10.15.19 1:30am - (Replying to @PHuenermund) I love how the "well defined" intervention disambiguates the do-operator.

10.15.19 1:13am - (Replying to @PHuenermund) Beg to differ. Here is an example of an explicit assumption that no one can judge harshly because no one understands: "Treatment is conditionally independent of the potential outcome, given covariate C." See how Imbens & Rubin (2015) struggle to explain what it means. #Bookofwhy

10.15.19 12:02am - (Replying to @analisereal and @edwardhkennedy) Gee, I can't hear what I said! Did I understand the question? Did I answer it adequately? I wish we had a professional transcriber in the audience, trained on Israeli accents and their variety -- machine learning 101. #Bookofwhy

10.14.19 7:51am - (Replying to @JadePinkSameera) Likewise. I crossed oceans in Muslim-Jewish dialogues and ate many Baklawas chanting our common Abrahamic tradition. My moment of truth came around 2009 when I realized that the idea of two ligitimate nations has not begun penetrating the mind-walls of my Palestinian partners.

10.14.19 7:23am - The Sept 2019 issue of the Journal of Causal Inference is now out and, as promised, I am linking you to the Table of Content My paper: "Sufficient Causes: On Oxygen, Matches and Fires" can be accessed here: #Bookofwhy

10.14.19 7:09am - Sharing a confession of an Israeli woman who shares my views about religion, people-hood, history and the prospect of peace.

10.14.19 2:03am - (Replying to @aselbst @mikarv and 19 others) That is what they argued against LaPlace "You will never know all the boundary conditions, give up" And that is why we invented probabilities, to summarize the unknowns w/o explicating them. #Bookofwhy

10.14.19 1:51am - (Replying to @MehmerPA @learnfromerror and 31 others) Jayne is forgiven for saying: "It's just our good old product rule", because he took P(A|B) as primitive. Historians will note however that there was no "|" symbol in Bayes days and writing the ratio P(A&B)/P(B) meant FORMALIZING the expression "given that we know B". #Bookofwhy

10.14.19 1:35am - Anti-Zionist bigots can't see the evil in their barking. It is not entirely their fault, for no one asks them to take a good look at the moral mirror. They will stop when Forward Magazine joins others in making Zionophobia the ugliest word in town. see

10.14.19 1:31am - (Replying to @EWilf) Anti-Zionist bigots can't see the evil in their barking. It is not entirely their fault, for no one asks them to take a good look at the moral mirror. They will stop when Forward Magazine joins others in making Zionophobia the ugliest word in town. see

10.14.19 1:10am - (Replying to @yudapearl @KyleCranmer and 6 others) I will try to make it to the reception Monday, and will pray to the Gods of meta-physics on Wednesday.

10.13.19 10:44pm - (Replying to @MehmerPA @learnfromerror and 31 others) Curious, who is the author of the paragraph you posted? The phrase: "it is really nothing but the product rule of probability theory" betrays his/her understanding of the rule. #Bookofwhy.

10.13.19 4:00pm - (Replying to @mikarv @aselbst and 19 others) Witch-doctors got structurally disempowered by modern medicine. Not a great loss. It is not legibility by computers that we should strive to achieve, but legibility by social scientists and social activists. Legibility by computers comes naturally once we understand ourselves.

10.13.19 3:45pm - (Replying to @KyleCranmer @ipam_ucla and 5 others) My, My, thanks for posting. I just discovered that I am scheduled to talk Wednesday pm. Is it an oversight? Or my forgetfulness?

10.13.19 3:17pm - (Replying to @aselbst @andrewthesmart and 19 others) If we wish to maintain hopes of changing an undesirable societal phenomenon we must create a scientific language to communicate about it, not leave it at the mercy of emotional obfuscation. #Bookofwhy

10.13.19 1:52pm - (Replying to @andrewthesmart @jainfamilyinst and 19 others) The language of causal diagrams does not stop you from "immersion in contexts" or from representing "dynamically changing constructs". It only stops you from leaving things hazy in your own mind or mis-communicating them to others. #Bookofwhy

10.13.19 3:31am - (Replying to @yudapearl @jainfamilyinst and 20 others) definition matches their conception of discrimination. The only requirement is that the definition be formal rather than hazy. In return, the diagram offers ways of applying the definition to data and estimating the degree of discrimination actually observed. #Bookofwhy

10.13.19 3:22am - (Replying to @jainfamilyinst @zhitzig and 19 others) Issues connected with Race and Discrimination invariably evoke pleas for re-definitions, without offering alternatives. Causal diagrams do not dictate one definition of discrimination. They offer a general representation of human behavior, and invite experts to propose whatever

10.13.19 12:01am - (Replying to @RonKenett) You may be right about the ignorance of our health-care colleagues at CUNY, but I try to refrain from using charges of anti-Semitism. Why give bigots an excuse to debate anti-Zionism vs. anti-Semitism when the former is morally more reprehensible and physically more dangerous.

10.12.19 7:58pm - Speaking of the history of Bayes Rule, my grandchildren found this cartoon, and complained that my beard is outdated #Bookofwhy

10.12.19 4:38pm - Thanks for sharing, @PHuenermund . I've always said econometric will one day reclaim its glorious past. Why? Because it was hatched in structural models, free of RCT imitations. Nowadays, while still under hypnotic spell of pseudo-experiments, restoration is inevitable.#Bookofwhy

10.12.19 4:09pm - And 50 years later, in 2019, a fake feminist named Linda Sarsour said: Zionism and Feminism don't go together. See

10.12.19 2:26pm - (Replying to @david_colquhoun @PeterMonnerjahn and 29 others) Agree on induction; beg to differ on randomization, as we say in #Bookofwhy:

10.12.19 2:12pm - (Replying to @JAdP) The idea of being responsible for someone's career change sends shivers of responsibility in my spine. I hope it was a positive change.

10.12.19 6:30am - (Replying to @cperek @harari_yuval and @sapinker) Yes, thanks for correcting.

10.12.19 6:28am - (Replying to @scottsantens @ElmSahd and 2 others) And I've left supporting Bernie Sanders when he chose James Zogby as Middle East "advisor". He is now flirting with Linda Sarsour and other Orwellian Zionophobes.

10.12.19 5:44am - (Replying to @yudapearl @JWSBayes and 32 others) The quote above was actually written by Edward C Molina, in his 1940 introduction to "Facsimiles of Two Papers by Bayes". Glenn Shafer's "On Bayes two arguments" (1982) is by far the best, and deserves a more elaborate summary (to appear) #Bookofwhy

10.12.19 5:34am - Sorry for the mis-link. The correct link is Enjoy the cosmos. #Bookofwhy

10.12.19 5:22am - (Replying to @JWSBayes @learnfromerror and 31 others) "The discussions and criticisms of Bayes' postulate have monopolized so many pages of the history of our subject that little, if anything, has been published on the most important feature of the essay." (Edwards Demmings, 1963) The best is Glenn Shafer's "On Bayes..."#Bookofwhy

10.12.19 4:55am - (Replying to @juli_schuess) Agree. Woodward never grasped the idea that SCM was designed to capture human knowledge, not one task or another. He himself uses SCM (DAG) as a basis for defining "interventions" and then poses "intervention" as a primitive. #Bookofwhy

10.12.19 4:12am - Sharing an illuminating interview with Yuval Harari @harari_yuval and Steven Pinker @sapinker which would lift your soul as it did mine: I use "soul lifting" in the AI sense: Take a long-term view of you place in the cosmos, & reset priorities.#Bookofwhy

10.12.19 3:23am - (Replying to @learnfromerror @f2harrell and 30 others) I am not surprised that folks are still disagreeing on what "being a Bayesian" means. But my question was more innocent: What did Bayes do in 1761 that caused this commotion? After all, he just equated two ways of writing P(A&B), which should have been equated before. #Bookofwhy

10.10.19 10:23pm - (Replying to @AndrewPGrieve @stephensenn and 29 others) Great question! First, unlike Bayes Rule, I cannot derive Pythagoras Theorem in one line. So, the question: "Whats the big deal?" does not apply. Second, Pythagoras Theorem has not been a topic of controversy by philosophers and practitioners alike. #Bookofwhy

10.10.19 10:09pm - (Replying to @f2harrell @stephensenn and 30 others) Student: What's the big deal? I can derive it in one line! Professor: Bayes is a way of thinking... Student: Thinking what? And why was this thinking not done before 1763? Nothing in probability theory prohibits it. I just derived it in one line!! Professor: ??? #Bookofwhy

10.10.19 6:07am - (Replying to @stephensenn @omaclaren and 29 others) Sure it is related. So, how do I explain to students why someone (ie., Bayes) became famous by proving a trivial identity. #Bookofwhy

10.10.19 5:16am - Thank you Raelle for remembering Danny's birthday. Indeed, he would be 56 years old today, traveling the world with his violin and laptop, spreading truth and understanding, refusing to believe that man is a predator of another man. Thanks to all who are joining us. HALLELUJAH!

10.10.19 4:26am - (Replying to @stephensenn @omaclaren and 2 others) It may be an excellent thing to do, but I am concerned only with whether doing so represents "what we know". I am yet to meet someone who can explain why one prior (on parameters of priors etc. etc...) is scientifically more plausible than another such prior. #Bookofwhy

10.10.19 3:58am - (Replying to @omaclaren @stephensenn and 29 others) Good article by Hajek, thanks! The relevant point is: "So while we are free to stipulate that `P(A|B)' is merely shorthand for this ratio, we are not free to stipulate that `the conditional probability of A, given B' should be identified with this ratio."(p. 101). #Bookofwhy

10.10.19 3:42am - Interesting observation. Except the 1948 abandonment of Israel was imposed by the U.N. while the 2019 abandonment of the Kurds is Trump's doing. I always tell Chechoslovakian colleagues we will never forget what their rifles did for us, targets of the 1948 "momentous massacre".

10.10.19 2:57am - (Replying to @stephensenn @smueller and 28 others) How would Dicing with Death change the argument that: P(A|B) is just a short hand notation for the ratio P(A,B)/P(B), hence Bayes Rule P(A|B)=P(B|A)P(A)/P(B) is a trivial identity. We need to explain how one becomes famous for deriving a trivial identity. #Bookofwhy

10.10.19 2:06am - (Replying to @omaclaren and @Prof_Livengood) Well put. If we "allow probabilities over whatever", like many Bayesians, then everything is "probabilistic", even Astrology, just assign Pr. to "unicorns have blue eyes" and other conclusions you desire to derive. Same as spraying priors on parameters (of priors...) #Bookofwhy.

10.10.19 1:54am - If you examine how "statisticians in the Neyman-Rubin-Holland tradition talk about causation by way of counterfactuals," you will see that they ignore principle (i) "It is silly to ignore what we know." Their assumptions reflect what they want, not what they know.#Bookofwhy

10.10.19 1:32am - (Replying to @katchwreck) This is the nature of basic research. But I agree with you that theoretical understanding of limitations should be part of every research endeavor.

10.10.19 1:21am - (Replying to @leskocar @andrearmolino and 2 others) Disagree here. Stratification and IPW are two different estimation methods, both requiring a valid choice of covariates from some graphical model, see Section 3.6. #Bookofwhy.

10.10.19 1:10am - (Replying to @leskocar @andrearmolino and 2 others) Agree with @leskocar . The answer depends on what the regression model is used for, a question that tends to surprise most regression modelers. I once suggested that we should take the "model" out of "regression models" because they do not model anything. Do they? #Bookofwhy

10.10.19 12:57am - (Replying to @fledglingStat) Yes there is a rigorous sense of deciding what control is good and what is bad, see: As to testing, the answer is NO, see "Why there is no statistical test for confounding..."(Causality chapter 6) #Bookofwhy

10.9.19 11:56pm - (Replying to @learnfromerror @stephensenn and 28 others) In which case P(A|B) is just a short hand notation for the ratio P(A,B)/P(B), and everything done in the name of Bayes can also be done in standard probability calculus, absent the "|" symbol. Is that a good summary of your position? #Bookofwhy

10.9.19 11:24pm - Had an invigorating Yom Kippur navigating a new sense of agency. Sang Mi-Haish in memory of our late son: "... keep you tongue from speaking falsehood..." and spoke on a panel on how we can turn Zionopobia into the morally ugliest word in town, along:

10.9.19 10:28pm - Dana Mackenzie and I are glad that one of the people who helped develop deep learning understands this limitation. We look forward to writing a second edition of #Bookofwhy where chapter ten will be replaced with success stories about how AI will overcome its causal blindness.

10.9.19 2:48am - (1/ ) Apologizing to readers for taking a Yom Kippur leave from Twitter. I do not fast. I do not observe religious rituals. Yet I, like many secular Israelis, observe Yom Kippur "religiously". I wish I could do so in Israel, where everything comes to a complete halt and "thinking" has
10.9.19 2:48am - (2/ ) a different meaning on that day. I bet our mind is driven by a different algorithm, relentlessly probing the essence of one's agency. #Bookofwhy

10.8.19 3:53am - An extremely valuable article on the influence of Wikipedia on science. Thanks. A possible bias: I often use Wikipedia to find citations to articles that I do not want to read. Sometimes to pacify a reviewer and sometimes to tell readers: "it's there, don't bother" #Bookofwhy

10.8.19 3:53am - An extremely valuable article on the influence of Wikipedia on science. Thanks. A possible bias: I often use Wikipedia to find citations to articles that I do not want to read. Sometimes to pacify a reviewer and sometimes to tell readers: "it's there, don't bother" #Bookofwhy

10.8.19 1:40am - Speaking of the spread of "fake waves" into our culture, there is no better example than the one coming this week from an actual "Minister of Culture": Note the straight face of an educated person (PhD) as he fakes history to fit an agenda.

10.7.19 7:45pm - And we were taught that mathematics is protected from the "fake wave". It was, until a colleague sent me this article: Now I only believe Bible stories, b/c I know the authors and the publisher.

10.7.19 4:49am - Wow! @jazchaz , I did not realize a hashtag #DanielPearlMusicDays exists! Now the whole world will sing "Mi Haish? Keep your tongue from evil and your lips from speaking falsehood (fake news?). Shun evil and do good. Seek peace and chase after it."(Psalm). Counterfactual-Hoping?

10.7.19 3:55am - (1/ ) Thanks @jazchaz for posting this photo of my son Danny, showing both his love for music and his determination to pursue truth and understanding. In two days we will be celebrating Yom Kippur (Day of Atonement), and I will be singing in his memory: "Who is the person who
10.7.19 3:55am - (2/ ) loves life?... Seek peace and chase after it" (Psalm 34: 13-15). It has become a logo for Danny's life and legacy. I am not as good as Hava Albertstein:, but I won't miss an opportunity singing it duet with a friend or stranger:

10.6.19 4:14pm - (Replying to @depistemology) I can see the connection now, thanks. Yes, "adjacent possible" is a good metaphor for the generation of counterfactuals (another is Lewis' "closest possible world"), except that, cognitively, we perform these local perturbations on a mental model of reality. #Bookofwhy

10.6.19 3:55pm - I see that Elias Bareinboim is speaking at Stanford again,, so, if you are around Palo Alto Monday 1pm and want to taste new wine from the causal brewery, it should strengthen your spine and improve peripheral vision #Bookofwhy

10.6.19 2:17pm - (Replying to @ganesh__s) Well put. I would only add that to appreciate what it does and does not do we need to simulate big data algorithms on toy models of reality, and test if we get the answers we expect. #Bookofwhy

10.6.19 2:06pm - (Replying to @depistemology) I'm guilty of missing Kauffman's theory. Would appreciate a pointer or, better yet, a Twitter-friendly description of principles and "input-output". #Bookofwhy

10.5.19 11:59pm - Ed Leamer's profound and entertaining 1983 classics is worth reading for the n-th time. My favorite slogan: "The mapping is the message", that is, the mapping from assumptions to inferences does not depend on assumptions. #Bookofwhy

10.5.19 7:13pm - Cheeting with bad controls is easy, as we have seen here: But it is getting harder and harder in the age of graphical models, as we can see here: #Bookofwhy

10.5.19 2:02pm - (Replying to @DrMoritzWvB @dompagano and 10 others) `People prefer to believe what they prefer to be true.' True, timely and still, this should not stop us from seeking truth and standing for what we believe to be true. #Bookofwhy

10.4.19 1:34pm - (Replying to @piamancini and @agentofuser) Glad you found causation interesting. Correlation is indeed so boring; strange how it kept us mesmerized for over a century. #Bookofwhy

10.4.19 6:18am - @thosjleeper , the high number of "likes" to your post encourages me to retweet it, because I realize that this crash course : is really fundamental for anyone using regression and wishing to "see" unbiased effects in one's own eyes. #Bookofwhy

10.4.19 2:53am - (1/2) As some of you know, October is when we celebrate -a global concert dedicated to "Harmony for Humanity". If you happened to be at Stanford (Oct. 23), Taipei (Oct.28), Santa Monica
10.4.19 2:53am - (2/2) or La Paz, you are invited to join the celebration and add an epsilon to world's sanity. There will be many more concerts but, unfortunately, due to malicious hacking, we can't upload them this year on our website.

10.4.19 2:18am - Sharing a recent paper on mediation analysis, with good introduction to path-specific effects: #Bookofwhy

10.4.19 12:17am - (Replying to @firstmn59 @mushfiq_econ and 2 others) Recall that the entire field called "econometrics" was authored by folks who did not speak DAGs, so the opportunities are enormous. Your topic should depend on whether your adviser is enlightened or old guard, and if your interests are methodological or domain specific #Bookofwhy

10.3.19 11:42pm - (Replying to @yudapearl @stephensenn and 29 others) Twitter is a great arena for bouncing multiple perspectives. I would love to learn from more folks how they explain why Bayes Rule is not a trivial identity, and what kind of conclusions it allows us to assert that we could not assert from straight probability calculus.#Bookofwhy

10.3.19 4:59pm - (Replying to @jlanastas @thosjleeper and @tmorris_mrc) We can see it in Carlos and Andrew's "A Crash Course in Good and Bad Control" - Models 3 and 5, for example, treatment (X) is selected by unobservables (U) yet treatment effect is identifiable by OLS. Similar for the napkin problem, etc etc #Bookofwhy

10.3.19 4:37pm - (Replying to @jlanastas @thosjleeper and @tmorris_mrc) "Selection on unobservables" is not a hindrance to identification. The phrase was probably coined by folks who did not speak DAGs (check me!) because DAGs do tell us what is and what isn't hindering identification. I heard economists are updating their lexicon. Kodos.#Bookofwhy

10.3.19 4:44am - (Replying to @stephensenn @smueller and 28 others) I would be delighted to announce that I've found a statistician who acknowledges the epistemological essence of Bayes Rule. What does Dicing With Death say about why Bayes Rule is not a trivial identity? Does it agree with #Bookofwhy page 102?

10.3.19 3:10am - (Replying to @smueller @david_colquhoun and 29 others) How can a trivial identity inspire so much reverence and controversy? I am yet to find a statistician who agrees that the essence of Bayes Rule is epistemological, i.e, it formalizes "given that we know". Most think epistemology belongs in psychology, not statistics. #Bookofwhy

10.2.19 9:44pm - I think it's worth retweeting.

10.2.19 6:55pm - (Replying to @mushfiq_econ @Jabaluck and @amitabhchandra2) Even the most tool-neutral philosophers cannot but notice that DAGs are second language in epidemiology and taboo in economics. Some attribute it to differences in research questions, some to types of data. Others, me included, to social & academic leadership. #Bookofwhy

10.2.19 2:52pm - (Replying to @learnfromerror @twainus and 24 others) @twainus , I would like to join this request, I tried but failed to understand your theories.

10.2.19 2:54am - (1/2) (Replying to @RonKenett @david_colquhoun and 29 others) There are so many labels around, that we must start talking substance (eg input-output), away from labels. What in havens is the purpose of "data integration" if one does not specify the needed output under diverse populations? As to the "domain adaptation" it has been around
10.2.19 3:05am - (2/2) (Replying to @yudapearl @RonKenett and 30 others) for at least 15-20 years and, according to my ML trusted colleagues, it has been given up as "undoable". Why? Because if we do not specify the reasons for data disparities, we can't fuse together heterogenious data properly. Stat. cant fuse Apples and Oranges.#Bookofwhy

10.1.19 10:48pm - (1/2) Thanks @jlanasta for the link to my new paper "Sufficient Causes: On Oxygen Matches and Fires" which will appear in the next issue of JCI. See: . I will post here the Table of Content of the entire issue, as soon as it is available. (Cont. )
10.1.19 10:48pm - (2/2) It is important especially to students entrapped in the PO tradition who cannot make the transition from rung 2 to rung 3 because, for them, Causal Effects emanate from counterfactuals, and counterfactuals require a "Treatment". Not so. #Bookofwhy

10.1.19 7:23pm - Totally different? I doubt it. Let's see. What type of descriptions do these "statistical methods" require about the disparate populations which they aim to "integrate"? As a part-time robot, I can't read Wiley books, but I can understand "input-output" #Bookofwhy

10.1.19 5:49pm - (Replying to @Lester_Domes @twainus and 11 others) Input-output specification is the only way to disambiguate the confusion of multi-meaning concepts like "information" "fusion" "integration". It works in the Ladder of Causation, and it should work elsewhere. Think robots to understand humans. #Bookofwhy

10.1.19 5:37pm - (Replying to @stevesphd @eliasbareinboim and 2 others) DAGs are there to help you precisely at this stage of your "real life" problem, by allowing you to explicate what you DO and DON'T know about that "life". Once you do that, you can view the DAG as a TOY, and turn to enjoy "real-life" again. #Bookofwhy

10.1.19 5:13pm - (Replying to @RonKenett @david_colquhoun and 29 others) As a former engineer (Technion 1960) I've learned the power of input-output thinking. So, What is the input needed to do "data integration" compared with "data fusion" (ala B&P) compared with "meta-analysis"??? Taxonomies based on input-output distinctions are useful #Bookofwhy others)

10.1.19 @9:12am - (Replying to @RonKenett @david_colquhoun and 29 others) I mention meta-analysis because Jeske and Xie mention it in their introduction to this special issue. (BTW, my definition of meta analysis: Averaging apples and oranges to get properties of bananas) Moreover, what is "data integration"? Formally speaking? #Bookofwhy

10.1.19 @7:34am - (Replying to @RonKenett @david_colquhoun and 29 others) To determine whether these methodologies are "much wider" or "much narrower" I would propose a litmus test: Solve the toy problems described in or . Meta Analysis is certainly helpless in dealing with causal disparities #Bookofwhy

10.1.19 @7:20am - (Replying to @JadePinkSameera @stephensenn and 29 others) Heretic? Me? In my circle of friends I am known as "defender of commonsense"

10.1.19 @7:00am - (Replying to @JadePinkSameera @stephensenn and 29 others) Watch the speakers list and, unless it contains one or two heretics, I doubt it would be interesting.

10.1.19 @6:55am - (Replying to @david_colquhoun @twainus and 28 others) Who is the offender? Where is the offense? Willing to defend innocence. #Bookofwhy

10.1.19 @6:50am - (Replying to @DavidSFink @amitabhchandra2 and @AdanZBecerra1) Well put. I am not sure about (b) but (a) certainly slows down conversations with economists. See my attempt to get them interested in answers to questions they themselves asked:, #Bookofwhy

10.1.19 @6:34am - (Replying to @GilmanStephenE @DavidSFink and 2 others) Not right! You have thus identified the causal effect of X on Y contingent on the assumptions that (1) A and B are the only confounders and (2) the system is linear. Contingent effects are weaker than absolute effects, but they can be VERY useful. #Bookofwhy

10.1.19 @5:58am - (Replying to @david_colquhoun @twainus and 28 others) Yes, its Nancy Cartwright, and she is also mentioned here: "Challenging the Hegemony of Randomized Controlled Trials" (Deaton and Cartwright 2018.) And in my comments: #Bookofwhy

10.1.19 @5:48am - (Replying to @amitabhchandra2 and @AdanZBecerra1) A DAG is a carrier of knowledge, hence I find it indispensable in thinking about MTE, or other properties of populations.Not alone, of course, but in combination with counterfactual logic. It is easier to educate those who misuse DAGs than those who misuse other texts #Bookofwhy

10.1.19 @5:29am - (Replying to @david_colquhoun @twainus and 28 others) In the language of causal logic, your contention is a "theorem" - no causes in, no causes out (Cartwright). The wisdom comes in answering: {Some causes in - which causes out?}. Once we solve this, we can go to: {Some causal assumptions in - which causes out?}. Not easy #Bookofwhy

10.1.19 @4:45am - (Replying to @david_colquhoun @twainus and 28 others) Thanks for linking me to this interesting conversation. I see two points over which I normally disagree with statisticians: Bayes rule and causality. The wisdom of causal logic lies not in assertions from data that don't warrant it, but from data that DO warrant it. #Bookofwhy

10.1.19 @2:45am - (Replying to @DrColinWPLewis and @d_spiegel) I love the quote "If the headline is interesting, it probably ain't true". @d_spiegel was also a pioneer in developing graphical models for expert systems (80's) so, if he says things are hard, he knows what he is saying, and he is ripe for joining the era of causation #Bookofwhy

9.30.19 @11:04pm - (Replying to @Jabaluck and @amitabhchandra2) No one is safe from bad assumptions - we are all humans. But making a bad assumption that is explicitly stated and can be communicated and repaired is better than concealing one's assumptions under cryptic names. I havn't mentioned "economists"-I know the sensitivities.#Bookofwhy

9.30.19 @9:31pm - I am retweeting this answer, because the question comes up in almost every interview that I give: "And what if we insist on using our Gold Standard, RCT, and none other!" Same question comes in ML context: "And what if you are a model-blind RL expert?" #Bookofwhy @amitabhchandra2

9.30.19 @5:42pm - (Replying to @AdanZBecerra1) DAGs are needed even if one vows to ban observational studies. Just to store and make sense of the results of several CRT's require some organizational structure (ie DAGs), not to mention fusing data across populations or correcting for selection bias. #Bookofwhy

9.30.19 @3:19pm - (Replying to @societyforepi @EpiEllie and 5 others) Can someone summarize: What does social epi bring to the table? The papers I could read do not tell you what. New kind of problems? New kind of tools? New discoveries? Curious.

9.30.19 @2:21pm - (Replying to @hcp4715 @psforscher and 3 others) I hope Primer did not reverse your faith in rung 2 and rung 3. As they say in the Talmud (or should have said) "commonsense is irreversible"

9.30.19 @4:12am - (Replying to @Kkushaj and @stanfordnlp) Plenty. See this thread:

9.29.19 @11:45pm - Now, that top quality educational material is available to statistics instructors, what do you think is the main hindrance keeping them from teaching causality in statistics? notation? mindset? If I was such instructor, I would jump at the opportunity. Do they? #Bookofwhy

9.29.19 @9:04pm - My sister in Tel Aviv reminds me that 2019 (5,779) also marks the 110 anniversary of the city of Tel-Aviv, where I happened to get a first peek of the cosmos. Who could imagine this miracle:

9.29.19 @3:41pm - The new year, 5,779-th from creation, is just an hour away and, as an AI student, I can't stop thinking: What an ingenious programming trick it is (for a robot): "Step back, observe the cosmos, take a tally, re-set priorities" and make believe it makes a difference. Shana Tovah

9.29.19 @7:05am - Congratulations! Julian, on co-winning the ASA "causality in statistical education Award", and sharing this interesting thread on how you entered CI from neighboring disciplines. I did not realize they selected winners in 2019. I hope you help others make the entry. #Bookofwhy

9.29.19 @4:50am - Sharing an insightful article by @vardi, touching on both: the early history of Neural Networks and the "Usefulness of Useless Knowledge" On page 111 of #Bookofwhy I describe how NN's influenced my research in the 1970's.

9.28.19 @11:01pm - (1/2) Welcoming the 2019 Jewish New Year (tomorrow), I wish all readers and followers, Jews and non-Jews, a Happy New Year, a year of peace and prosperity, progress and understanding, in science and in the global human condition. As a piece of history, I am sharing this top view
9.28.19 @11:01pm - (2/2) of the stones of the Western Wall, in Jerusalem: which reminds me how I saw those stones for the first time, a 3 years old, and how my mother said: "This is why I came to this country." It was 1939, when her family was stranded in Poland, by the Nazis.

9.28.19 @12:57am - (Replying to @dan_marinazzo @manlius84 and 12 others) Whether causal inference is tricky cannot be seen in because this text is paralyzed by "ignorability" blinders and, oddly, skips modern developments that make causal inference fun and transparent. For contrast, see #Bookofwhy or

9.27.19 @8:36pm - This edited version is an improvement, but my favorite video is still the one I gave in Paris: (Microsoft, 2012) perhaps because the ideas were still new to me, fresh from the brewery, or because I was 7 years younger. #Bookofwhy

9.27.19 @2:11pm - Who is doing the work, and who can spoil the work of others is shown visually in Carlos and Andrew's "A Crash Course in Good and Bad Control" -A road map guide for the perplexed traveler, novice and seasoned. Enjoy! #Bookofwhy

9.27.19 @2:02pm - (Replying to @ProfMattFox) Your thought is valid. Adjustment for a parent (surprisingly) does not "partially adjust" for the child. This is shown in, section 3.2 and explained in Appendix I. #Bookofwhy

9.27.19 @5:04am - An improved video of my talk at the Why-19 symposium is now available here: It provides captions of the questions asked during the Q&A sessions (press the CC button), as well as copies of the slides used. Welcome to the CI-framework. @Bookofwhy

9.26.19 @7:01am - I assume you are as embarrassed by Dr. Mahthir Mohamad as many Malaysians are, and you are offering an excuse for his mentality. The excuse, however, is more embarrassing than the offense, for it denies one people the right to self determination. It begs for a major revision.

9.26.19 @5:34am - An interesting paper on causal mediation analysis from Political Science perspective: Tracing Causal Paths from Experimental and Observational Data X Zhou, T Yamamoto - 2019 #Bookofwhy

9.26.19 @3:33am - Yesterday we saw a creepy performance by Mahthir Mohamad, PM of Malaysia, who told the world: "I am proud to be called an anti-Semite", and made the whole world wonder: Why would a Prime Minister of a respected country expose the level of education he received in that country?

9.26.19 @3:07am - Thank you, @juli_schuess for creating this thread. It is both an honor to the winners and an invitation to statistics educators to enrich the perspectives of their students with new tools and new understanding. The class material is all there: Books, slides and notes. #Bookofwhy

9.26.19 @1:08am - Your personal invitation to a Causal dance. A video of my Stanford Why-19 talk is now available here: Apologizing for the faint audio during Q&A, but at least you can see the slides regardless of where the speaker stands. Enjoy the dance of WHY #Bookofwhy

9.25.19 @3:53pm - (Replying to @NeuroStats @learnfromerror and @PHuenermund) My "strange birds" comment referred to interest in "causal inference", not in "graphical models". Still, I remain open minded, curious to examine 2020 statistics textbooks, plus next ASA presidential address on the frontiers of stat research. #Bookofwhy

9.25.19 @12:25pm - (Replying to @learnfromerror and @PHuenermund) There are a few converts, e.g., Lauritzen , Dawid, Didelez. But they are considered strange birds in mainstream. #Bookofwhy

9.25.19 @3:49am - This new book: Handbook of Graphical Models captures well how the field has grown since its inception in the 1980's. Written by top statisticians, it can serve as a good introduction to causal modeling for traditionally-trained statisticians. #Bookofwhy

9.25.19 @3:03am - And I would challenge any of my peers to convince us that the racist words that Malaysia PM Mahathir Mohamad is about to spew at Columbia today are just a matter of "political sensitivity", not a human right issue, and of no concern to true "scientists".

9.25.19 @2:37am - (Replying to @patrick_s_smart @ForecasterEnten and 3 others) HMM, now we know what political scientists do in their spare time. Causality cannot explain this link, but it can help reconcile all these interesting opinions on what makes one factor a better predictor than another. I can see how they can benefit from a causal graph #Bookofwhy

9.25.19 @2:16am - (Replying to @gomezramirez_ac) Scientists are very much like entertainers. They rely so heavily on peers approval that one can rarely tell what they truly stand for. Some even use the cover of "no-politics" to justify their over-sensitivity to peer approval. We need more thoughtful straight-shooters #Bookofwhy

9.24.19 @5:52pm - (Replying to @gomezramirez_ac and @f2harrell) I do not understand what you deem to be at odds with scientific ethos. My scientific ethos was authored by people like John McArthy, who threatened to cancel an AI conference in Tbilisi (1975), unless soviet dissidents get permission to talk.

9.24.19 @5:34pm - (Replying to @exactsake and @f2harrell) Of course I support the ban, this is what the Massachusetts initiative is all about.

9.24.19 @5:31pm - (Replying to @f2harrell) It's the price one has to pay for trying to be 1. helpful, 2. thoughtful, 3. truthful and 4. principled. When scholars are being told to shut up, that's when politicians take charge, and scholars come complaining: "How? On my watch?" Yes, on your watch! #Bookofwhy

9.24.19 @1:45pm - (Replying to @f2harrell) Students at Columbia and other followers expect my support on this Twitter handle, and they consider their plight to be a moral imperative, not "political views". I concur, and I can't let them down.

9.23.19 @11:13pm - I join Columbia University students call on President Bollinger to issue a denunciation of Dr. Mahathir Mohamad, saying: "I can't stop you from speaking, but it's my duty to tell you that you are not welcome in this University. Your values are not ours."

9.23.19 @10:50pm - (Replying to @malpaso) Having been brought up in Mogadishu, Omar surely met a few victims of FGC, and many Minnesota voters have hoped that she would raise hell in Congress against the practice and spearhead petitions like the one in Massachusetts. My point is to make people wonder: Why hasn't she?

9.23.19 @8:49pm - (Replying to @cmirzayi) Having been brought up in Mogadishu, Omar surely met a few victims of FGC, and many Minnesota voters have hoped that she would raise hell in Congress against the practice and spearhead petitions like the one in Massachusetts. Has she?

9.23.19 @5:06pm - My! My! We are talking Massachusetts 2019, not Mogadishu 1600. I bet Congresswoman Ilhan Omar signed this petition; she was born there, in Mogadishu.

9.23.19 @4:56pm - (Replying to @omaclaren and @HenningStrandin) Good question! Formal Causal Inference is still in its Embryonic formative stage, with tiny influence in academia, industry and other power houses, so it is still protected from the dangers of dogmatism and out-datedness that mature disciplines should watch for. #Bookofwhy

9.23.19 @2:55pm - (Replying to @HenningStrandin) This immediately brings to mind: statistics, economics and machine learning but, given the professional-personal sensitivities involved, it is risky business to bring anything to mind. So lets just quote Venn (1834-1923), as if it does not pertain to modern sciences. #Bookofwhy

9.23.19 @2:44pm - (Replying to @gescher) One of my favorite quotes. #Bookofwhy

9.23.19 @5:04am - Sharing an answer posted on Quora to the question: "Why is it that machine learning systems are black-boxes." It touches on the two notions of "explainability". #Bookofwhy

9.22.19 @11:53pm - (Replying to @Sisyfuzz) No. I am not familiar with this docuseries. Bill Gates was instrumental in pushing Bayesian Networks in Microscope, but I have not heard his current opinion on the causal revolution. Any pointer? #Bookofwhy

9.22.19 @10:26pm - (Replying to @EllieAsksWhy and @wtgowers) But then, how do you handle a new structure, that does not fit into the one you learned VERY well? If you go through at least one structure-smashing experience in your life, you develop the muscles to view structures as tools, not sanctuaries. #Bookofwhy

9.22.19 @10:02pm - (Replying to @EWilf and @UNWatch) May the bells of history inspire you as you stand up for Human Rights and commonsense. And don't forget to use the key word, Zionophobia, which tells HRC delegates: Recall, your children will one day be reading your words from the record, would you be able to bear the shame?

9.22.19 @4:20pm - (Replying to @mpbennett @Josh_Bersin and 2 others) I still do not see there the distinction between explaining the program vs. explaining the world. Consider: Q. "Why the increase in crimes?" Ans. "Because we hired more policemen" #Bookofwhy

9.22.19 @3:58pm - (Replying to @osazuwa) Good point. The difference between SCM and PO however is not that great, since both are about "world outcomes" not "program outcomes". The latter, is wedded to "treatments", the latter to "undoing of past events", not necessarily "treatments" of "deliberate actions" #Bookofwhy

9.22.19 @6:13am - (Replying to @wtgowers) We need indeed the more standard paths, but I would not worry about depleting their rank; the incentives to remain within the bubble far exceed those given to the cross-field explorers. #Bookofwhy

9.22.19 @5:27am - (Replying to @mendel_random) Indeed, Haldane's quote crossed my desk from your inspiring tribute to Jerome Cornfield, concluding: "Perhaps one of the advantages that Cornfield had was his lack of any sustained formal training in either epidemiology or biostatistics". Lesson: To impact stat. - study history.

9.21.19 @10:40pm - (Replying to @AvigailFerdman) Hitcharatnu?

9.21.19 @10:01pm - A brilliant quote crossed my desk which I could not let go un-shared (from Haldane JBS. , Science and life , 1969): "I consider it desirable that a man's or a woman's major research work should be on a subject in which he or she has not taken a degree." #Bookofwhy

9.21.19 @7:54pm - (Replying to @PHuenermund) I was waiting to hear: "... they take you to a special room & tell you how to get an academic appointment in another econ. department but ...". At great physical risk...

9.21.19 @6:18pm - (Replying to @EWilf) From my explorations, the idea of two-states has not begun to scratch the surface of their heads. I searched for it in what Palestinian intellectuals write on Israel's independence day

9.21.19 @7:13am - This philosophical paper,, titled "Instantiating Sapience", describes #Bookofwhy as "An engineering approach to AI." I must confess that I cannot understand the many non-engineering alternatives described in the paper, perhaps one of our readers could.

9.21.19 @5:51am - Beware of an important distinction between "counterfactual explanations" in ML, eg., and in science. The former explains when "system output" would change. The latter explains when "world outcome" would change. #Bookofwhy

9.21.19 @4:36am - (Replying to @quantumciaran) Oh, I looked at the wrong paper. It is that deals with the instrumental inequality, thanks. I was not aware of these extensions and ramifications in quantum systems. #Bookofwhy

9.21.19 @4:27am - (Replying to @quantumciaran) Fascinating paper. But it is not clear to me which of your results generalizes the instrumental inequality of ?

9.20.19 @3:41pm - (Replying to @EWilf) Speaking of Zionism and "displacement," I always find it illuminating to read a page or two in the history of the Yishuv : and listen again and again to the clash between the "Me, Me, Me!" and the "We, We, We!"

9.20.19 @4:32pm - I dont dig this theater. It seems like a waste of money to fund a billion $ Apollo program without knowing what the moon looks likes, or how far it is. I would rather pass another mini-Touring test (as in #Bookofwhy), or build a robot with sparkling free-will.

9.20.19 @2:40am - (Replying to @smueller) I think the culture that produces Israeli researchers was created by the Zionist pioneers of 1917-1947 who came there without their professors - a pre-requisite to creative thinking. (Hard to do in America.) #Bookofwhy

9.20.19 @2:04am - (Replying to @quantumciaran) Thanks for the link. Indeed, the causal discovery perspective makes sense here: What structure can explain the quantum correlations predicted (and empirically confirmed) by Bell. They also touch on the smart & under-utilized IV-inequalities #Bookofwhy

9.20.19 @1:37am - It was a pleasure reading your review of #Bookofwhy, partly because you confessed the pleasure of reading it, and partly because you described it from a fresh perspective, uncharged with preconceived molds that insiders usually carry. I posted it on

9.20.19 @1:37am - It was a pleasure reading your review of #Bookofwhy, partly because you confessed the pleasure of reading it, and partly because you described it from a fresh perspective, uncharged with preconceived molds that insiders usually carry. I posted it on

9.19.19 @8:31pm - As an American researcher, I am worried about US standing in the Spectator Index. As a former Israeli, I am surprised at the low number: 8250. In my home town, everyone, even the butcher and the grocery man were accomplished researchers -- they always knew all the answers.

9.19.19 @7:23pm - The literature on Bell Inequality in quantum physics has been growing fast in recently years, but remained fairly cryptic to ousiders. This paper turns things around: If you know causal models, you can understand Bell Inequality. #Bookofwhy

9.19.19 @3:39pm - (Replying to @ZahraBilloo) @ZahraBilloo You just dont get it. It is Zionophobia that is growing and dangerous, as is Islamophobia and so much more. Spell it: Zionophobia - The obsessive animosity against a homeland for the Jewish people. Please spell it. @WomensMarch @ZionessMovement

9.19.19 @8:22am - Glad you asked, the definition is simple: Zionophobia -- An obssessive animosity toward the idea of a homeland for the Jewish people. I have defined it in more details here:

9.19.19 @7:32am - Greatest news since I woke up! A Zionophobic bigot identified and voted out. But the @WomensMarch will remain a fake movement until they add a member of @ZionessMovement to their national board. After all, don't they speak for ALL women? Have they forgotten feminism's pioneers?

9.19.19 @6:58am - (Replying to @rickwahs and @melb4886) They might, but they don't, because mathematics overcomes terror through its usefulness. Every high-school kid notices it when terrified in the first class of algebra. Try it on your favorite research question! #Bookofwhy

9.19.19 @6:49am - (Replying to @rickwahs and @melb4886) Indeed I remember this book and, now, going over the chapters, I can't help but thinking how clearer many of them be had the authors used the mathematical language of Beebee Hitchcock and Price. Day and night. Some of the chapters are still in the dark age. #Bookofwhy

9.19.19 @5:48am - (Replying to @rickwahs and @melb4886) Please teach us what is incremental about the dramatic shift of philosophers' writings on causation, from informal to mathematical, and what "other people", beside those recognized and revered in #Bookofwhy we should credit for this shift. Eager!

9.19.19 @5:35am - This picture convinces me that democracy has a life of its own, detached from egos and personalities, and that, against all odds, history may be on the side of this tiny democracy. Lets hope.

9.19.19 @5:00am - (Replying to @rickwahs and @melb4886) Good question. Could the answer be that many people are interested in learning new things rather than judging things or people from a distance? Or could it be that most people, after learning it, understand that causal inference is not a incremental passing hype? #Bookofwhy

9.19.19 @4:41am - (Replying to @AdanZBecerra1 and @AlexBroadbent) This "end of the day" "practical setting" aspiration, and how it is informed by the do-operator is shown here: #Bookofwhy

9.19.19 @4:36am - (Replying to @p_realism and @davidpapineau) I met @davidpapineau in a conference late 1980's and I remember that his lecture was different from those of his fellow philosophers. Can you remind me of his main contribution? #Bookofwhy

9.19.19 @12:52am - (Replying to @AlexBroadbent and @AdanZBecerra1) Vanderweele does not go all the way, as I explain here:

9.19.19 @12:45am - (Replying to @EsserHartmut @litgenstein and @tomdrabowicz) I am still eager to know what principles, methods or tools this literature has spawned that we, lay persons would be deprived of, if not read. What kind of causal problems can they solve or formulate that we have neglected to address? #Bookofwhy

9.18.19 @11:20pm - (Replying to @EWilf) For years I was wondering if foreign journalists really don't see the formula, or just play blind, to keep the conversation going. I now know; its the former. Why? b/c to see it you need to read a chapter or two in the history of the conflict and dig the red lines of the 2 sides.

9.18.19 @10:36pm - (Replying to @PHuenermund @eliasbareinboim and 2 others) I am interested in those who are NOT using DAG methods. Can they do any meaningful fusion beyond taking weighted averages of two or more sources? #Bookofwhy

9.18.19 @10:27pm - (Replying to @EsserHartmut @litgenstein and @tomdrabowicz) I must have missed this literature on Explanative Sociology". Do you know if any of the tools presented in #Bookofwhy owes its existence to this literature? I hope its not too late to make up for the oversight, in case I failed here

9.18.19 @10:13pm - (Replying to @FabItMart and @causalinf) The self-referencing is in your mind. I am pointing to a dramatic shift in philosophers writings, from informal to mathematical (crediting Wright), which was surprising to me. By fearing self-referencing, you risk missing the benefits of the shift. Enjoy them first. #Bookofbook

9.18.19 @8:38pm - (Replying to @AdanZBecerra1) I can only answer such questions if told the principles by which a package does what it says it does. For a quick answer, if it does not use do-calculus, I doubt very much a package can solve the simple examples presented here Please check. #Bookofwhy

9.18.19 @8:27pm - (Replying to @FabItMart and @causalinf) This is precisely what #Bookofwhy does. It starts with the Garden of Eden, and goes to great pain of explaining the difference between "having interest in causation" and "mathematizing causation". What are you (and others) objecting to? Eden? The Garden? or the mathematization?

9.18.19 @7:28pm - For anyone who is interested, Carlos is going to demo the "Causal Fusion" software tomorrow, during his talk at the SoCal 2019 Methods Conference, UC Riverside. You can see the papers and the agenda at: #Bookofwhy

9.18.19 @7:13pm - (Replying to @FabItMart and @causalinf) It goes back even to the Garden of Eden. See #Bookofwhy chapter 1. Who does not have interest in a form of thinking that governs everything we know and do in the world.

9.18.19 @7:08pm - (Replying to @Prof_Livengood @learnfromerror and 6 others) Great bibliography. Refreshing and proving my point. #Bookofwhy.

9.18.19 @4:47pm - (Replying to @yudapearl @learnfromerror and 6 others) Its hard to swallow. As I discussed on Amstat News Someone must be over-generalizing, it's truly hard to swallow. #Bookofwhy.

9.18.19 @3:36pm - (Replying to @litgenstein @EsserHartmut and @tomdrabowicz) "Interest of philosophers" goes back to Democritus and earlier. Even statistics emerged from "interest in causality". Who doesn't have such interest?" I should have said "interest of philosophers in modern causal inference". Thanks. #Bookofwhy

9.18.19 @3:28pm - (Replying to @learnfromerror @JadePinkSameera and 5 others) No I do not overgeneralize. When I wrote "show no interest" I did not mean "have no interest", I meant "show"!! In writings, in meetings, etc. Can we count the number of times the word "causal" appeared in your recent summer symposium? Why deny "show no interest"?#Bookofwhy

9.18.19 @3:14pm - (Replying to @litgenstein @EsserHartmut and @tomdrabowicz) Certain ideas are surely new in that book, others are old. I am holding Hume (1739) and Mill (1843) in my hands(literally) and, for the life of me, I can't see how they could resolve Simpson's paradox or other puzzles. Could any pre-1990 philosopher? Why deny progress? #Bookofwhy

9.18.19 @2:54pm - (Replying to @JadePinkSameera @StatModeling and 5 others) I am of the impression that such forum would be boring, because the differences, commonalities and miscommunications have been rehashed to exhaustion in the literature and Twitter. Note also that, sadly, "philosophers of statistics" show no interest in causal models. #Bookofwhy

9.18.19 @7:45am - (Replying to @yotambarnoy) I am aspiring to the same ending, hoping that acting as equal will end their separate and hostile existence.

9.18.19 @4:32am - (Replying to @EsserHartmut @ingorohlfing and @tomdrabowicz) In social science, following Blalock Duncan et al, the ideas of testing and identifications are not new. But if you were a student of Suppes or Lewis, these ideas are new, b/c you cannot identify the effect of something (eg action) that you cannot represent. #Bookofwhy

9.18.19 @4:17am - While we are at it, here is another book on experimental philosophy, also dounlaudable, with a chapter (11) on "causal reasoning". #Bookofwhy

9.17.19 @11:58pm - An unexpected opportunity emerging from the Israeli election.

9.17.19 @11:32pm - Philosophers of science are beginning to take interest in causal inference. This new book (downloadable) shows how traditional problems in philosophy come to light through the new lens #Bookofwhy

9.17.19 @3:08pm - Sharing a nostalgic photo. Some place in my army days (1953-1956) I remember driving a truck, collecting ballots from remote army units, to be counted. A most heroic endeavor on the roads of those days. Our mission: Not one soldier left uncounted. It worked!

9.17.19 @2:36pm - (Replying to @jprwg @strangecosmos and 2 others) Knowledge (or "information") is what constrains your answers and drives them from "maybe" to yes/no or probable. I am reluctant to use "information" because people tend to confuse it with Shannon's information - a purely probabilistic notion. "Mechanism" is too narrow #Bookofwhy

9.17.19 @2:10pm - (Replying to @on_clusters @ABravoBiosca and @IGLglobal) The walls of NBER are taller than the Himalaya. No access to anyone but club members.

9.17.19 @1:47am - (Replying to @hofbeezy) Not only doesn't it hurt, it is also a beautiful poetry; like chanting Kiddush or singing Hatikva. Its not the words but the melody.

9.17.19 @1:28am - (Replying to @strangecosmos @GaryMarcus and @ylecun) If by "ML techniques" you mean any future algorithm, then you are justified in saying "I dont know". But if by "ML techniques" you mean algorithms based on data only, we can tell even today that the answer is NO. We can't compute 3-dim volume from a 2-dim shadow. #Bookofwhy

9.17.19 @1:17am - (Replying to @RivasElenaRivas) Let's do it together. Fit a 100-layer NN to data coming from ice-cream sales and crimes. Interpret the fitted NN as structural causal model and ask it: "Would crime increase if we ban ice-cream?" What answer would we get? #Bookofwhy

9.17.19 @12:54am - (Replying to @strangecosmos @GaryMarcus and @ylecun) Why speculate on what Sutton means or meant? Do you @strangecosmos believe that any ML technique can solve any of the toy problems in #Bookofwhy or Primer, given all the data in the world, and no information beside data?

9.17.19 @12:43am - Today, it is an election day in Israel. Public transportation is free, from anywhere to everywhere. I wish I could take one of those trains and fulfill my duty (I have dual citizenship). For a deep understanding of issues and moods see

9.16.19 @10:18pm - (Replying to @womensmarch) @womensmarch has just replaced a fake feminist with a confessed Zionophobe. Is there anything more weird, fraudulent and contradictory than an anti-Zionist feminist?

9.16.19 @10:11pm - (Replying to @SethAMandel)

9.16.19 @10:10pm - (Replying to @Marissa_Jae)

9.16.19 @9:57pm - (Replying to @haileybanack @EpiEllie and @AmJEpi) Just in case you wish to inflame the paradox with more fuel, here is a gallon from my own brewery In #Bookofwhy we took the liberty of calling the birth-weight phonomenon a "paradox", though it has been explained. Paradoxes enjoy cultural immortality.

9.16.19 @9:22pm - Retweeting my comment on a controversial change in the leadership of the Women March movement. An interesting thread.

9.16.19 @8:40pm - (Replying to @ZionessMovement) I just can't agree with your last sentence, implying that "anti-Zionism" is less racist than antisemitism. The former is not only bigoted but also eliminationist and borders on genocidal. Are you implying that anti-Zionism is the lesser of the two evils?

9.16.19 @8:25pm - (Replying to @ZionessMovement) Women March, Inc. has just replaced a fake feminist with a confessed Zionophobe. Is there anything more weird, fraudulent and contradictory than an anti-Zionist feminist?

9.16.19 @6:09pm - (Replying to @strangecosmos @GaryMarcus and @ylecun) There are theoretical impediments that even "general methods" cannot circumvent. If you search for a common point of two parallel lines, you can have the best search-and-learning method in the world and you won't find one. ML confronts such impediments to reaching GAI. #Bookofway

9.16.19 @4:22am - (Replying to @ewerlopes) Yes. Causality is heavier, with all the proofs, and histories, and arguments with philosophers and economists and statisticians. Primer is game-like: "Look Ma, I can do today what I couldn't yesterday, and it makes so much sense!" You dont want to miss it. #Bookofwhy

9.15.19 @6:38pm - So, perhaps I was too naive in assuming that my colleagues in the stat dpt have been doing non-parametric estimation for the past two centuries. But now, that they are awaken to the importance of causal estimands, do they need outside help? #Bookofwhy

9.15.19 @6:04pm - Agree. But why do we have to "think hard" if the task of estimation has been the sole target of super smart statisticians for over a century, backed by an empire that produced thousands of PhD's. Are we "smarter" then they? or are CI estimands new to them? #Bookofwhy

9.15.19 @4:32pm - (Replying to @LauraBBalzer and @GilmanStephenE) By all means, assuming someone younger does the writing, and someone older does the diplomacy. #Bookofwhy

9.15.19 @2:43pm - (Replying to @GarridoWainer) No official document, just a long discussion on Twitter, followed by a survey whether it is ethical for an author to publish the review he/she received. I believe majority voted YES. #Bookofwhy

9.15.19 @2:35pm - After #Bookofwhy, the next entry into causal inference is the PRIMER It was praised already by so many readers, so I won't add, except to note that Wiley is coming up with a clean version next month. In the meantime, the corrected chapters are accessible.

9.15.19 @2:23pm - (Replying to @GilmanStephenE and @LauraBBalzer) By the HUGE gap, do you mean going from a finite sample to an estimate of the estimand?

9.15.19 @4:36pm - (Replying to @manuelbaltieri) To make the Ladder of Causation more connected to "thinking" and cognitive functions, I was considering labeling the rungs: 1. Foresight, 2. Control. 3 Understanding. It rings better with Toulmin, 1961, "Forecast and Understanding" #Bookofwhy

9.14.19 @7:52pm - (Replying to @vardi) I wonder what Melinda Baldwin's opinion would be on our proposal to make all reviews public, 5 years after decision (anonymously if requested), so as to remind reviewers of the higher judgment of history. #Bookofwhy

9.14.19 @7:52pm - (Replying to @vardi) [You Retweeted] I wonder what Melinda Baldwin's opinion would be on our proposal to make all reviews public, 5 years after decision (anonymously if requested), so as to remind reviewers of the higher judgment of history. #Bookofwhy

9.14.19 @4:05pm - (Replying to @PHuenermund) I see nothing wrong in self-citing, especially if no one else has articulated the idea that you deem relevant. The problem I do see is that most self-citings point to irrelevant publications, bearing little relationship to the discussion in the text. #Bookofwhy

9.14.19 @3:50pm - Some symbolic gifts fill your heart with gratitude, but thinking what this gift would do to the spirit of the rockets-stricken children of Sderot, stops your heart from beating.

9.14.19 @2:29pm - In ancient Greece, "over-Democracy" led to the invention of formal logic; they had to reign-in the endless arguments. What invention will Israel's democracy lead to? A crucial election will take place on Tuesday. Stay tuned.

9.14.19 @10:56am - In this paper on Off Policy Evaluation I was particularly interested in Appendix C: Bridging the Gap between Reinforcement Learning and Causal Inference. #Bookofwhy

9.14.19 @10:43am - Another causal discovery paper . This one applied to diabetes data. #Bookofwhy

9.14.19 @10:33am - This Economics Bulletin paper applies causal discovery to future prices in the Chinese Stock Index 300. It seems someone is about to get rich soon. #Bookofwhy

9.13.19 @1:26pm - (Replying to @manuelbaltieri) I trust readers of #Bookofwhy are equipped with night-vision glasses that tell them right away that no definition of causality can be constructed in probabilistic vocabulary, no matter how sophisticated or skillful.

9.12.19 @7:12pm - (Replying to @katchwreck) By enlarge, the literature on "dimensionality reduction" stands orthogonal to causality, by virtue of being statistical. There is however a point of contact when we minimize the number of covariates we need to adjust for. Beautiful algorithms exist for this task. #Bookofwhy

9.12.19 @4:23pm - Every time a reader praises Primer I take a minute and read a paragraph or two, and come back with an urge to reply: "You are so right!". Today I succumb to this urge and recommend it to all readers, especially free Ch4, #Bookofwhy

9.12.19 @4:46am - (Replying to @harrydq and @TelegraphTech) Many thanks for this pointer. For the life of me, the last things we contemplated was getting embroiled in the Brexit debate. On the other had, if the WHY rules the world, why should this debate be excused. #Bookofwhy

9.12.19 @3:50am - (Replying to @TelegraphTech and @harrydq) Too bad I was blocked by the Pay Wall as it became interesting. #Bookofwhy

9.11.19 @11:07pm - As 9/11 day comes to an end, I am sharing one of the most profound requiems I have ever heard: "Kaddish para Daniel" Written by Benjamin Lapidus, it combines Hebrew, Aramaic and Spanish in a rythm that shakes the foundations of our souls. Kaddish for 9/11

9.11.19 @10:02pm - (Replying to @BarryOSullivan and @UCC) A milestone in the history of AI.

9.11.19 @8:31pm - (Replying to @BarryOSullivan and @UCC) Wow! And my copy of Boole's Laws of Thoughts (1854) has the signature of James William Warren, AM, MRIA, Sep. 1864. Was he a professor at Cork? We, book collectors, form a bond of ownership. A weak form of immortality.

9.11.19 @8:08pm - (Replying to @BarryOSullivan) UCC!! Cork Ireland!! Thats where George Boole wrote The Laws of Thoughts (1854) and Boolean Algebra was born!! Glad you are still marking the pre-Christian calendar. I do too!! Though mine is in pre-Christian Hebrew, going (logically) from right to left. Cheers, and many springs

9.11.19 @5:27pm - It is still 9/11, and it is LA, my home town, where a Temple of ex-Morroco Jews has been vandalized. The populist slogan "Free Palestine" has become a license for every disgruntled group to spread its "My-Grievance-Above-All" mentality wherever it clicks.

9.11.19 @4:44pm - The first victim of 9/11, Danny Lewin (Z'L), was an internet innovator and a graduate of my alma mater, the Technion in Haifa, Israel. He was on board AA Flight 11 from Boston to LA, and was murdered as he struggled to advance toward the cockpit. Watch

9.11.19 @12:43am - An opportunity to learn DAGs from Felix Elwert, one of causal-inference top teachers: . #Bookofwhy

9.11.19 @12:14am - It's 12 midnight, September 11, in LA, an hour I can't forget, b/c my son was murdered by the same people who made this day unforgettable. I salute the people of Israel who remember my fellow Americans through these columns of lights in the "9/11 Living Memorial", in Jerusalem.

9.10.19 @5:49am - This paper on "counterfactual fairness" has reached my desk: and reinforced my conviction that "fairness" is a counterfactual notion, and must hence be managed by structural models - the breeding grounds of counterfactuals. #Bookofwhy

9.9.19 @10:04pm - History enthusiasts will probably find the discovery of this colorful mosaic to be a proof that man is a history-seeking machine. The place, Tabgha, is were I spent some of my army days, laden with sweet nostalgic memories of the sea of Galilee, Kinneret in Hebrew.

9.9.19 @5:04pm - I have added a link to Maudlin's review of #Bookofwhy on and, following our lively discussion here, I've added comments to clarify some not-so-obvious points in the book, especially the difference between Rung Two and Rung Three in the Ladder of Causation

9.9.19 @2:42pm - Our AI-minded readers should find this NYT interview with Gary Marcus and Ernest Davis to be illuminating: #Bookofwhy

9.8.19 @4:49am - (Replying to @PHuenermund and @nickchk) An important distinction that makes the difference between those who can take a causal problem and bring it to a stage where it can be estimated by random trees and other statistical tools and those who forget the first stage and assume it was prepared by someone else. #Bookofwhy

9.7.19 @4:46pm - (Replying to @autoregress) Sorry it I am/was going after the wrong economists, but think about what an enlightened econ. student feels upon hearing from leaders in his/her field: "We are waiting to be shown the money". Is this the spirit of Haavelmo, Marschack and Arrow? #Bookofwhy

9.7.19 @4:30pm - (Replying to @soupvector and @aregenberg) Thanks for posting, and welcome to this Twitter Zoo, where you can meet other readers enjoying their ability to do things they always wanted to do, including convincing economists that they, too, can do things today they always wanted to do. #Bookofwhy

9.7.19 @4:23pm - (Replying to @autoregress) "I like my flashlights demonstrated in the real world!" but I won't try them myself, not even near my faithful lamppost, and if someone demonstrates them elsewhere, I say: "We are different, for us .... for us "real world" is what we read in good econometric journals.#Bookofwhy

9.7.19 @4:00pm - (Replying to @autoregress) The "flashlight" works around your IV lamppost as well, see Fig. 5.1 in Causality (2000), you just need to press on the right button. #Bookofwhy

9.7.19 @3:53pm - (Replying to @autoregress) Imagine how many lost wallets and "good experimental studies" are awaiting their owners in train stations while economists cling to one lamppost, and won't try a flashlight, even one that performs well in epidemiology and other train stations. #Bookofwhy

9.7.19 @2:47pm - To all my Brazilian readers, colleagues and students - Happy Independence Day.

9.7.19 @2:38pm - (Replying to @HananyaNaftali) Thank you @HananyaNaftali for helping me start the day on a positive note. ps. I am trying to contact you by email:

9.7.19 @2:15pm - (Replying to @jon_y_huang @AdanZBecerra1 and 7 others) "will never accept" is a pretty bold statement for a discipline that prides itself on inventing structural approaches. As a student of history, I am fascinated by what happened to those economists, and whether modern day econ. students will shake them away from the new lamppost.

9.7.19 @1:50pm - (Replying to @DorotheaBaur) Not only "he doesn't entirely agree with the ladder" he actually misses the key transition from Rung-2 to Rung-3, as I explain in an earlier thread:

9.7.19 @12:58pm - (Replying to @DorotheaBaur) Thanks for posting, but note that "Pearl seems to think they are loaded with philosophical significance" is too humble. The #Bookofwhy actually claims that they are essential in science, & of practical significance in legal, medical and policy decisions.

9.7.19 @3:33am - (Replying to @sapinker) My first reaction to Maudlin's review was: The profound separation between interventions and counterfactuals is shown in (App. I), and by "Mute!" I mean: It would surprise me to find an idea from pre-1990 philosophy that I missed.

9.6.19 @12:38pm - (Replying to @djinnome) You are right, thanks for noting.

9.6.19 @12:35pm - (Replying to @_Srijit) Correct. Like "token" vs. "type", individual vs. population, actual vs. average.

9.6.19 @5:58am - (Replying to @_Srijit) What you and Maudlin are missing is remembering that interventional studies are non-deterministic estimating averages over populations (or unknown factors.)The statement "knowing that I am going to win" is not what the study gives you. I discuss it here:

9.6.19 @5:36am - (Replying to @yudapearl @arturlsc and 6 others) I am surprised that the computational aspects of DAGs is so underestimated. DAGs permit us to answer questions which otherwise are intractable. E.g.,"Tell me if the partial correlation R_{XY.Z} is zero", or "Tell me which parameter is estimable by OLS" #Bookofwhay

9.6.19 @4:08am - (Replying to @arturlsc @Jabaluck and 5 others) You must be kidding. Can you name another representation scheme, out of the many others, which allows you to see the testable implications of your causal assumptions ? #Bookofwhy

9.6.19 @3:02am - (Replying to @asweinmann) Appreciating your kind words. After a week of arguing with on-lookers, it is refreshing to hear from an unspoiled reader, interested in science, not in arguments. #Bookofwhy

9.6.19 @2:27am - Echoing our discussion of interventions and counterfactuals, I have summarized part of it in this paper submitted to the next issue of JCI. Comments are welcome, pointing to omissions, disagreements and improvements; my deadline = Sep 12 #Bookofwhy

9.6.19 @12:58am - (Replying to @Jabaluck @PHuenermund and 4 others) You have very low opinion of experimentalists, assuming they are incapable of generalizing from a toy problem to patterns of impediments they see in their substantive works. #Bookofwhy is written for enlightened experimentalists who, once aware, will dismantle those impediments.

9.6.19 @12:42am - (Replying to @Jabaluck @PHuenermund and 4 others) Every modeling task assumes away many difficult things, but at least we can represent them explicitly, and reason about ways of overcoming those "most difficult parts." I have not seen this ability demonstrated in "mostly harmless". #Bookofwhy

9.6.19 @12:33am - (Replying to @Jabaluck @PHuenermund and 4 others) Beauty! Please teach us the vocabulary of that "own language" and its internal logic, perhaps it is more effective than DAGs, since quasi-exps. think it is more "reliable", and has resulted in a "credibility revolution". I am truly curious, as computer scientist should #Bookofwhy

9.6.19 @12:19am - (Replying to @yudapearl @Jabaluck and 5 others) And what's the added wisdom? Are the impediments to fitting identification strategies in "real-life studies" different than 1. confounding,2. non-exclusion, 3.selection bias etc. all of which are representable in "make up" examples if one is serious about handling them.#Bookofwhy

9.6.19 @12:09am - (Replying to @Jabaluck @PHuenermund and 4 others) Please show us one challenge that cannot be represented in a made-up example and that only reveals itself in "real-life studies" where, instead of symbols, variables are decorated with: "return to school" "return to prison" "Years in service" What's the big deal?#Bookofwhy

9.5.19 @10:36pm - (Replying to @PHuenermund @Jabaluck and 4 others) @PHuenermund please remind me what a tDAG is. Is it a mental representation of knowledge which "quasi-experimentalists" consult when fitting an identification template, and which they refuse to commit to paper for fear of appearing traditional, and exposing assumptions?#Bookofwhy

9.5.19 @10:26pm - (Replying to @Jabaluck @PHuenermund and 4 others) Your skepticism reveals how powerful DAGs are in unveiling assumptions that DAG-averse cultures hide under the rug, ensuring that no one ever questions the confidence with which exogeneity or exclusions are overstated in quasi-experiment. The hidden invites no question#Bookofwhy

9.5.19 @10:01pm - (Replying to @Jabaluck @PHuenermund and 4 others) What about "DAG-invented" strategies like "conditional IV" or "Instrumental set" or "bow-free" or even Fig. 5.10 ?? Some are "DAG-invented" and some are "DAG-synthesized". The jury is not stupid, it's just kept ignorant by tribal zealots. #Bookofwhy

9.5.19 @8:48pm - (Replying to @Jabaluck @PHuenermund and 4 others) I surely believe that DAG as a good working hypothesis about the process (with Friedman's critics of secondary importance) and, more importantly, as a model for a huge number of real-life cases in which the mediator is physically "shielded" from the confounder. #bookofwhy

9.5.19 @8:35pm - (Replying to @Jabaluck @PHuenermund and 4 others) And to clarify, I do it too. If you look at the text below Fig. 5.10, I search the DAG for patterns of identification and, when not found, I repair the ones that are repairable. Except, I do it on a drawn DAG, not on my mental DAG, which I believe to be less reliable. #Bookofwhy

9.5.19 @8:21pm - (Replying to @Jabaluck @PHuenermund and 4 others) This is how Phillip and Sewall Wright "stumbled upon" IV's, now a celebrated "research design". They started with 4-5 variables, causal relations among them, Sewall's trick of converting them to covariance constraints and faint hope of inverting them back, no "design". #Bookofwhy

9.5.19 @8:07pm - (Replying to @RashidaTlaibz) Strangely! I do not see any Jewish Stars among the marchers! What happened to my old comrades? Sad! @EWilf

9.5.19 @7:31pm - (Replying to @y2silence) Interesting! Does Rosenbaum use the words "cause of crying"? Rubin proclaimed such questions to be "more of a cocktail conversation topic than a scientific inquiry", thus purging the word "why" from the vocabulary of his disciples. I thought his verdict still reigns. #Bookofwhy.

9.5.19 @7:14pm - (Replying to @_Srijit) Causal Bayesian Networks are called "Causal" because, unlike ordinary Bayesian Networks, which are purely associational, they provide answer to all interventional questions: What if we raise taxes? ie., all policy related questions - just what economists should rejoice #Bookofwhy

9.5.19 @7:02pm - The coin-game (below) exemplifies this kind of contradiction. Win-on-correct-guess is one model of the world, random winning (ignoring your guess) is another. Both are compatible with experiments, yet the second says:"No, you would't have lost had you acted differently"#Bookofwhy

9.5.19 @5:43pm - I owe readers an explication of what I mean by: "Intervention studies CANNOT ANSWER counterfactual questions". "Cannot answer" means that two different world models, both compatible with the studies, can generate two contradictory answers to the same question. #Bookofwhy

9.5.19 @2:19pm - Glad you re-posted this explanation of the difference between rung-2 (intervention) and rung-3 (counterfactuals) in the Ladder of Causation. Many researchers still find it hard to swallow (eg Maudlin) especially RL folks, for whom the world is just "interventions"#Bookofwhy

9.5.19 @5:59am - For a simple example consider a game where we win upon guessing the outcome of a fair coin and lose otherwise. The action "guess head" has no effect on winning, neither has "guess tail". Yet, upon winning, we can assert: "Had we acted differently we would have lost".#Bookofwhy

9.5.19 @5:47am - To elaborate, Causal Bayesian Networks [Causality Ch. 3] enable us to compute the effects of all possible actions, compound actions and actions conditioned on observed covariates and, still, none can answer the couterfactual:"What if we have done things differently? #Bookofwhy

9.4.19 @10:16pm - (Replying to @Jabaluck @PHuenermund and 4 others) "Real-world" again? Take any one of those revered "quasi experiment" successes in which exogeneity and exclusion were contested. Done. But I thought the discussion revolved around what comes first, the DAG or the ident. strategy. So now we start with an oracle, no DAG.#Bookofwhy

9.4.19 @9:42pm - (Replying to @yudapearl @PHuenermund and 5 others) "Starting with nothing" means no DAG, no SEM, no IV, just an oracle that can tells you, for every variable that comes to your mind, what the "sources of variations" are for that variable. #Bookofwhy

9.4.19 @9:20pm - (Replying to @PHuenermund @Jabaluck and 4 others) I tried to stay out of this discussion because it went over my head. I think @Jabaluck methodology could become clearer if he tells us how one should handle the problem in Causality Fig. 5.10 (below), step by step, starting with nothing but the desire to estimate beta.#Bookofwhy

9.4.19 @1:05pm - I just read Maudlin review. It's largely sensible, save for two mistakes: (1) ".. it is not possible to think causally but not counterfactually." Causal Bayesian Networks demonstrate that it is possible. (2) I was intimately familiar with the Tetrad project of the 1980's. Mute!

9.4.19 @5:53am - (Replying to @stephenpollard) This precious lady is so saintly innocent, that I begin to believe she really does not understand why people would call her a racist for supporting a racist movement. Many BDS supporters can't stomach it: Me? A racist? See

9.4.19 @3:00am - (Replying to @PHuenermund @pierre_azoulay and 4 others) What is the canonical example(s) economists used to demonstrate "selection on observables"? What are students told to do in such cases? #Bookofwhy

9.4.19 @2:54am - (Replying to @saurabh_jha21) Causal inference goes beyong ml/dl models, so I do not think you can implement the former with the latter.

9.4.19 @2:52am - (Replying to @FelixThoemmes and @y2silence) I heard about the Venn diagrams used in the context of "variance explained". Can you explicate what intuition they support?

9.4.19 @1:26am - (Replying to @y2silence) I interpret Horst's surprise (upon finding a suppressor in the data, 1941) as evidence that regression analysts expect correlations to behave like separation in graphs, that is, if a node Y is separated from X and from Z, it must also be separated from the pair (X,Z). #Bookofwhy

9.3.19 @11:11pm - Should have used it as a trailer for #Bookofwhy. Don't knock it, the script writer was a thoughtful philosopher.

9.3.19 @2:42pm - (Replying to @y2silence) And the winner is Yongnam Kim @y2silence !!! The example I had in mind had S and X interchanged, which works just as well, and tells us that either S or X need to be a collider for a suppressor to play tricks on us. Now, back to Horst (1941): Why was he surprised? #Bookofwhy

9.3.19 @1:40pm - (Replying to @FelixThoemmes) Keen observation. But the example I have in mind has no cancellation, and is familiar to every UG student of probability or statistics. #Bookofwhy

9.3.19 @1:36pm - (Replying to @djinnome) The independencies stated are not shown in the graph. eg X and S are shown dependent. So they need to rest on some compelling process.

9.2.19 @11:45pm - Hate to keep you in suspense. Yes! A super-suppressor does exist! Its a variable S, uncorrelated with X and Y, that, if added to the regression, turns X from a useless to a perfect predictor of Y. Can readers guess who S is? His name will tell us what suppression is. #Bookofwhy

9.2.19 @2:09am - (Replying to @matt_vowels) On the other hand Horst (1941) was interested in prediction, something I did not realize. Evidently, prediction-minded people also have intuition. It comes, I surmise, from causal assumptions that sneak secretly into intuition about predictions. Deserves some thought. #Bookofwhy

9.2.19 @12:39am - For readers who wrote to me last month about anti-Semitism and anti-Zionism, this latest article by Gil Troy is the best I've ever read: And Weiss's book that Troy reviews: is an insightful, eye-opening microscope of our generation.

9.1.19 @11:46pm - Social Scientist: Look what I found! A suppressor! Statistician: Big deal, it shows in regression analysis. Computer Scientist: Why were you surprised? Statistician: This is a question for psychologists. Social Scientist: No, its a question for all scientists Why was I surprised?

9.1.19 @11:35pm - To understand "suppressors", it is instructive to examine a "super-suppressor": A variable S that is uncorrelated with the regressor X and with outcome Y, yet, when added to the regression equation, turns X from useless to perfect predictor of Y. Puzzle: Does S exist?#Bookofwhy

9.1.19 @1:41pm - (Replying to @NunezKant) Agree. We sometimes forget what science is all about, and it is amazing that 12 years after, I still need "words" and "caps and gowns" to be reminded. Thanks for re-posting. #Bookofwhy

9.1.19 @4:59am - An unprecedented development at the UN. First time this world body addresses the inner core of the Arab-Israeli conflict, which was discussed earlier on this Tweeter. Kudos to the Brazilian delegate.

9.1.19 @3:25am - Today, September 1st, marks the 10-year anniversary of the publication of Causality (2009, 2nd ed.) I am proud to see that the book has stimulated 14,850 citations on Google Scholar and that, oddly, I am still agreeing with everything it says. #Bookofwhy

9.1.19 @2:40am - For readers who are tired of listening to my Israeli accent, here is my incredible co-author, Dana Mackenzie, introducing #Bookofwhy in plain English and lucid eloquence.

8.31.19 @5:05pm - (Replying to @nbarrowman) "Contributing cause" stands between "sufficient" and "necessary" cause. Note this interesting Wash. Post. definition of "Analysis": Interpretation of the news based on evidence, including data, as well as anticipating how events might unfold based on past events. All predictive!

8.31.19 @12:19pm - (Replying to @yudapearl @ang_hermann and 3 others) Specifically, the probabilities that annotate arrows emanating from "action nodes" in a decision tree are P(y|do(x)), not P(y|x), as classical textbooks might suggest. The former need DAGs to be estimated. #Bookofwhy

8.31.19 @11:55am - (Replying to @ang_hermann @furtadobb and 2 others) DAGs ARE used in the theory of decision. The reason we estimate P(y|do(x)) is to "Find x that maximizes E[U(y)|do(x)]" where U(y) is the utility of outcome y. For use in decision trees, see

8.31.19 @11:43am - (Replying to @TimFooler) No, I haven't given any such thought, worth looking into, while keeping in mind what we want to know and what we do know, ie, input--> output.

8.31.19 @1:45am - (1/3) This is an excellent paper, that every regression analyst should read. Primarily, to appreciate how problems that have lingered in decades of confusion can be untangled today using CI tools. What I learned from it was that the "suppressor surprise" is surprising even when
8.31.19 @1:45am - (2/3) cast in a purely predictive context: "How can adding a second lousy predictor make the first a better predictor?" Evidently, what people expect from predictors clashes with the logic of regression slopes. The explanation I offered here (Section 3)
8.31.19 @1:45am - (3/3) shows how the phenomenon comes about, but the reason for the clash is still puzzling: What exactly do people expect from predictors, and why? #Bookofwhy

8.30.19 @12:45am - (Replying to @analisereal @reflecmec and 4 others) One should add here that the second kind of intervention is identifiable whenever ETT is (ie, effect of treatment on the treated), as demonstrated in Primer (p. 109-111) primer-ch4:, #Bookofwhy

8.30.19 @3:58am - (Replying to @ClaudeAGarcia) And they dare tell us they have hard time recruiting subjects for randomized treatment ..... Hoping you have a great summer.

8.30.19 @1:24am - (Replying to @Moshe_Hoffman) Very interesting thread, touching on a long debated concept "the actual cause" [Causality ch. 10]. An important distinction may illuminate your analysis: "necessary vs. sufficient" causes. A recent post demonstrates it in the Oxygen-Match story #Bookofwhy

8.29.19 @11:28pm - (Replying to @boredyannlecun) According to Von Neumann we should all research computers. Thermostat controls are an xxx trillion $$$ industry. Counter that fact! #Bookofwhy

8.29.19 @9:45pm - (Replying to @analisereal @thosjleeper and 3 others) Great, you just proved that "linear models" imply homogeneous effects, but not the other way around. Linear combinations of nonlinear functions also guarantee effect-homogeneity. [Assuming that by "effects" we mean differences eg E[Y|do(x1)]-E[Y|do(x2)] @Bookofwhy

8.29.19 @8:02pm - (Replying to @thosjleeper @DanielNevo and 3 others) This has been my consistent usage, Yes. Although I would not be surprised if someone discovers a "nonlinear model" exhibiting the "effect homogeneity" property. #Bookofwhy

8.29.19 @7:56pm - (Replying to @ildiazm and @mgaldino) Agree. Parametric regression models comes in two varieties: 1. Carriers of statistical assumptions, eg. "E[Y|x] is linear in x" and 2. Statements of estimation strategies, eg. "Find the best linear estimate of Y, given x, regardless of the actual shape of E[Y|x]". #Bookofwhy

8.29.19 @1:57pm - (Replying to @DanielNevo @ildiazm and 2 others) I believe @analisereal summarized the "linear model" issues fairly well here: A distinction between "linear in parameters" and "linear in variables" is highly warranted. #Bookofwhy

8.29.19 @5:16am - (Replying to @bwundervald @nickdaleburns and 3 others) Good try, but the nonlinear function x3=x1*x2 makes the model nonlinear. It is not a matter of convention; it is substantive. In linear models, all causal effects are the same for all units (ie, all values of the error terms) We cant change this property by renaming. #Bookofwhy

8.29.19 @2:59am - (Replying to @nickdaleburns @RduvalH and 2 others) A "linear function of the separate predictors" is nice and explicit. But the words "linear model" may lead to some confusion, for the reasons I mentioned.

8.29.19 @2:48am - (Replying to @davekarpf) OK, now that you are a national celebrity, can you explain to us mortals what were you bedbugs a metaphor for? What did you mean to say in that comparison? Who are your heroes and your bedbugs?

8.29.19 @2:07am - (Replying to @thosjleeper and @mgaldino) Interesting. I was not aware of this confusion. Any reference to a Social Science book using this nomenclature? (preferably by authors who know the difference between regression eqs. and structural eqs.)

8.29.19 @1:50am - (Replying to @thosjleeper and @mgaldino) I am eager to learn, which community is it that labels a regression equation with product terms a "linear model"? Who are the careless authors who would do so? Why would they do it? #Bookofwhy

8.29.19 @1:41am - (Replying to @causalinf and @Undercoverhist) As tribute to the great fun we had today, I would like to dedicate a day each month to meet with potential authors of economics textbooks. Authors better be: 1) Tenured, ie free of peer pressures 2) Aspiring to brighten up the dark sides of econometric education #Bookofwhy

8.29.19 @12:27am - (Replying to @mgaldino) Sorry if any confusion, but I naively assumed that a regression equation containing a product term would not be classified as a "linear model". Two reasons: 1. it is not linear. (2)It is not a "model" of reality (ie a carrier of assumptions), but a tool of estimation. #Bookofwhy

8.28.19 @5:18am - (Replying to @emc2G @DanzigMD and 3 others) And you buy into this racist propaganda? Would you use the "separate but equal" metaphor on any other 2-states, say US & Canada? or US & Mexico? This comparison to racial segregation was manufactured by enemies of coexistence, and people in your sphere of information buy it?? SAD

8.28.19 @5:05am - (Replying to @emc2G @DanzigMD and 3 others) You make it sounds like "subjugation" is an Israeli pastime recreation, rather than a predicament forced upon them by neighbors who openly declare their intentions. Don't you read what Palestinians tell their children?

8.28.19 @2:55am - (Replying to @tdietterich) I would be very interested in your collection, when done. I think scientific creativity and theory formation are structurally similar to improvisational problem solving, as both invoke "modular template breaking" like the Lion Man in #Bookofwhy Chapter 1.

8.28.19 @1:24am - (Replying to @DanzigMD @emc2G and 3 others) I, for one, never understood what people are trying to achieve by saying the conflict is "COMPLICATED". An excuse from solving it? A diversion from addressing its core? To me, it's baby simple: A clash between two legitimate national movements, one says WE, the other says ME.

8.27.19 @9:45pm - (Replying to @emc2G @DanzigMD and 3 others) The comparison may sound inaccurate if you do not consider Zionophobia a form of racism, with genocidal intentions. I do, for the reasoned explained here:

8.27.19 @8:06pm - (Replying to @emc2G @DanzigMD and 3 others) @DanzigMD compared David Duke racism to the racist rhetoric and activities of Rashida Tlaib. Ashrawi and Miftah are more sophisticated, they know how to apologize once the damage is done. I have not heard Tlaib apologize for her Zionophobic outbursts. She can't! betray her base.

8.27.19 @2:57pm - (Replying to @emc2G @DanzigMD and 3 others) You would never get a racist to admit to what he/she is. The litmus test: Charge them with Zionophobia and see how proud they sing.

8.27.19 @5:40am - (1/2) A bunch of new papers have reached my screen which seem related to discussions we have had here on tweeter.
[PDF] Transcriptomic Causal Networks identified patterns of differential gene regulation in human brain from Schizophrenia cases versus controls [Three more...]
8.27.19 @5:40am - (2/2) [PDF] Data Management for Causal Algorithmic Fairness
Counterfactual Reasoning for Process Optimization Using Structural Causal Models
[PDF] Reinforcement Learning is not a Causal problem #Bookofwhy

8.26.19 @11:33pm - @RepSchneider has stood up to @IfNotNow with "shades of gray", but I prefer Joe Biden's answer to the same bullies (paraphrased): "Occupation! Occupation! I have not met a single Palestinian leader who is willing to accept Israel's right to exist". Its black and white! No gray!

8.26.19 @9:51pm - (Replying to @f2harrell @EpiEllie and 6 others) This article was written 2008 and, yet, I see no sign of causal definition of "diagnosis". I wonder if probabilistic notions of diagnosis are still ruling the field? Or have they been replaced by Diagnosis = Best Explanation, as in ??#Bookofwhy

8.25.19 @10:47pm

8.25.19 @10:47pm - I had the privilege of knowing Danny Cohen in the 1990-1980's and of watching his brilliant mind at work. Unlike us, back-seat academics, he was an adventurer in real life - a pilot, a fighter, a system builder and a tough skeptic of AI. He will be missed. #Bookofwhy

8.25.19 @10:18pm - (Replying to @ayusharms) In more advanced studies one need indeed to accommodate cycles, (See for example Causality p.215 ). DAGs however allow us to leverage the full power of do-calculus, which has not been matched yet in cyclic systems. #Bookofwhy

8.24.19 @5:23pm - (1/ ) (Replying to @Chris_Auld) 1/I guess what you take as "formally similar" I take as vastly dissimilar. In one case (X-rest.) I can immediately write down the OLS estimand of EVERY parameter and in the other (cov-rest.) it is still an open question whether some parameters are identified, awaiting a decision
8.24.19 @5:38pm - (1/ ) (Replying to @yudapearl and @Chris_Auld) of whether those parameters can be solved uniquely from the covariance matrix [!!! decision] The mystery may be dissolved if you can just walk my students by the hand in Fig.5.10, & starting with the 3 eqs., show them why adding W-->Z spoils beta and W<--Z does not. #Bookofwhy

8.24.19 @5:01pm - (Replying to @Chris_Auld) Where discussion has gone? I am trying to extract the set of principles that leads econ. students towards the solution of Fig. 5.10. So far, a failure. Along the way, you said "cov. restrictions are just extension of exclusion restrictions" which blew me off 2 miles. #Bookofwhy

8.24.19 @4:39pm - (Replying to @Chris_Auld) My glitch! I meant the effect of Z on X is not identified. The same as delta in Fig. 5.10. Same as beta, if arrow W-->Z is added. In contrast, with exclusion restrictions alone (& all eps's uncorrelated), all parameters are OLS identified; not 2SLS, but straight OLS. #Bookofwhy

8.24.19 @12:39pm - (Replying to @Chris_Auld) Here is something it cannot do:
with Eps(Z) correlated with Eps(X), and Eps(Z) uncorrelated with Eps(Y) By "cannot do" I mean "it cannot identify effect of Z on Y." Moreover, it cannot tell us in general which system of eqs. is identifiable. #Bookofwhy

8.24.19 @6:52am - (Replying to @Chris_Auld) I dont see why covariance restrictions are straight forward extension of exclusion restrictions. The latters permit all effect to be identified by OLS, the former are still an open problem. Why is Z exogenous? Dont we need to examine the Z eq.? eg. what if we add W-->Z #Bookofwhy

8.24.19 @1:48am - European readers may have interest in this workshop on causality by @RonKenett . I hope he will share the slide with us, so we can learn what "fishbone diagrams" can do for CI, and how business applications can benefit from the causal revolution. #Bookofwhy

8.23.19 @6:33pm - (Replying to @Chris_Auld) I am not disputing the necessity or advantage of doing algebra vs. other methods. I am just asking: What are econ. students taught to do in cases like Fig. 5.10 ? No traps to my question. Just trying to learn something I could not get from the econ. literature. Help! #Bookofwhy

8.23.19 @3:39pm - (Replying to @Chris_Auld) Agree. But how is this "determination" done? By symbolic algebra? (super exponential) or, by step by step reasoning, as in Causality p. 153? #Bookofwhy

8.23.19 @3:28pm - (Replying to @analisereal) So, for my understanding, given model 5 below, DR will instruct us to adjust for both U and Z, correct?

8.23.19 @3:13pm - (Replying to @oacarah @AndersHuitfeldt and 4 others) The content probably add clarity, but the abstract speaks of protecting "consistency", not "precision" which, to me, adds to the confusion. #Bookofwhy

8.23.19 @10:04am - (Replying to @Jsevillamol) Thank you for making confounders simple (is it an oxymoron?) in this post. I would suggest another warning, against proxies of mediators, as demonstrated here: #Bookofwhy

8.23.19 @8:26am - (Replying to @AndersHuitfeldt @AdanZBecerra1 and 3 others) Thanks for making this distinction clear. Curious, why has "double robustness" developed in the context of causal inference tasks and not in classical statistics? Or has it? #Bookofwhy

8.23.19 @2:31am - (Replying to @jon_y_huang @AdanZBecerra1 and 2 others) What is the simplest model to demonstrate this preference? Doesn't it violate the heuristic advocated in: "A Crash Course in Good and Bad Control" #Bookofwhy

8.23.19 @1:42am - (Replying to @AdanZBecerra1 @jon_y_huang and 2 others) I understand that "double robustness" provides protection against misspecification in your model. Thus, naturally, your model (ie DAG) needs to be consulted before deciding if protection is needed and, if so, what protection would be adequate. #Bookofwhy

8.23.19 @12:41am - Watch this video of my friend @DanzigMD about Congresswomen @Ilhan & @RashidaTlaib & how they were caught partnering w Miftah, a Zionophobic NGO that accused us of using "the blood of Christians in the Jewish Passover."

8.22.19 @11:47pm - (Replying to @mehdirhasan) One thing you would never get a Zionophobe to accept: that anyone else has a right to dignity and self determination. They win debates on anti-semitism, but will never debate the ugliness of Zionophobia.

8.22.19 @10:46pm - I believe my guiding mantra would not be inappropriate here: "Only by taking models seriously we learn when they are not needed". #Bookofwhy

8.22.19 @10:23pm - (Replying to @josephpapptheat @ADL and 2 others) I believe by "our trauma" they mean the Zionophobic bigotry of Rep. Rashida Tlaib and the way some Democrats embrace this bigotry.

8.22.19 @2:47pm - (Replying to @FJnyc @mehdirhasan and 2 others) The new Orientalism: Mehdi Hasan is defining Jewish identity. And CNN pretends he knows what he is talking about.

8.22.19 @12:54pm - (Replying to @Chris_Auld) I am unfamiliar with the evaluation method you mention. How would your students tackle Fig.5.10 ?? What would be the first step? Input --> output? #Bookofwhy

8.22.19 @12:14am - (Replying to @JohannesTextor @PHuenermund and 2 others) Interesting paper on ranking efficiency of adjustment sets in linear models. I presume this coincides with the ranking in for non-parametric models. A skeptical economist may still ask: What about 2SLS? My answer: Where do econs. see 2SLS? #Bookofwhy

8.21.19 @10:41pm - (Replying to @n_iccolo and @FriedrichHayek) This is the crucial first step. Next, all you need is to read Primer and, believe me, you will be way ahead of most ML folks. #Bookofwhy

8.21.19 @9:46pm - (Replying to @yudapearl and @Chris_Auld) A question to economists and other folks curious about the role of "reduced form equation" (RFE). Q. If I were to ask 100 econ students what the RFE's are in this model: would I get one answer? three answers, or ten answers? Can we see one? #Bookofwhy

8.21.19 @9:29pm - (Replying to @FriedrichHayek) This is indeed an awesomely important work, that requires at least a few days/weeks to digest, with the aim of sorting out what primitive causal templates infants possess, and how advanced causal structures evolve from those templates. #Bookofwhy

8.21.19 @7:07pm - (Replying to @PHuenermund @djvanness and @Chris_Auld) True, DAGs are not expected to talk about efficiency, nor does any model of the world, they nevertheless do, to the maximum extent that efficiency considerations are dictated by the world. Are the alternatives more informative about efficiency? @Bookofwhy

8.21.19 @6:58pm - (Replying to @analisereal and @Chris_Auld) My earlier tweet seems to have gotten lost from this thread. Here is is And I wish a seasoned economist would tell us how his/her students are taught to solve this problem, step by step. #Bookofwhy

8.21.19 @6:18pm - (Replying to @tytung2020) The two laws are described concisely on page 168 here

8.21.19 @6:47am - Ten years ago I wrote this survey paper, which provides a panoramic view of the various approaches to causal inference. Its aim was to unify, rather than differentiate. I believe it was successful in showing how they all emerge from two laws. #Bookofwhy

8.21.19 @4:18am - (1/ ) It was not the mean-spirited tone of "uninformed" that triggered my reaction, but the realization that so many well-intentioned people can psych themselves into believing that slogans are knowledge. One of the reasons that I often fear left tyranny as much as right tyranny is
8.21.19 @4:18am - (2/ ) that the former strives to base its claims on "universal knowledge". "Everyone knows that there was no Temple in Jerusalem" said Arafat to Clinton. "Everyone knows that Israel is 'apartheid' state" chant BDS cronies, even intellectuals. I flip by the sound of those chants.

8.21.19 @1:39am - I haven't seen this intriguing thought experiment before. It is amazing how much truth can be unveiled through a mechanical transposition of just two words Jewish<---> Muslim. It highlights how crucial counterfactuals are to scientific thought. #Bookofwhy

8.21.19 @12:41am - (Replying to @lewbel) I thought non-parametric models include non-additive and non-separable errors. If true, then exog. ensures the OLS identifiability of RFE [Assuming, of course, we agree on what exogeneity and identifiability mean]. Oh God, when will we have a consensual glossary??? #Bookofwhy

8.20.19 @11:50pm - Thanks for noting the connection between Causality and the search for truth. I have explained here why I decided to continue both on @yudapearl . Someone has to counter the sirens of deceit that continue to dishonor the floor of the US Congress.

8.20.19 @11:28pm - (Replying to @DSPonFPGA and @SoniaCuff) Always a thrill to read ancient papers, like reading Greek mythology, Gee, what we used to believe in those days! #Bookofwhy

8.20.19 @3:45pm - (Replying to @Chris_Auld) Do we have a list of requirements that would guide my students towards an econ. solution of this problem? Something in the form: 1. examine the equations, 2. check if there exists...such that... 3. next check... #Bookofwhy

8.20.19 @3:38pm - (Replying to @Chris_Auld) This is a very interesting observation, deserving a mention in 3rd edition. The DAG only says: "use either beta1 or beta, both are consistent". The efficiency comes from noticing that beta1 invites 2sls est., which is not in the DAG, but in the mind of the modeller. #Bookofwhy

8.20.19 @5:23am - (Replying to @Chris_Auld) According to the definition in Phil Haile slides, Z is not exogeneous, as it is correlated with W. Can you outline just the conceptual steps of how LII is achieved. #Bookofwhy

8.20.19 @5:04am - Retweeting Elias talk on "Causal Data Science." If I were in Boston, I would not miss it. But given the circumstances, we will wait for the utube video. #Bookofwhy

8.20.19 @3:20am - (Replying to @CalumDavey) Appreciate your applaud but, for me, this is not "politics." It is a matter of being true to myself and my identity. It is paying back to a community that has invested dearly in my education and is now unable to fend for itself under this new barrage of populist slogans.

8.20.19 @2:39am - (Replying to @melvinwevers) Would "singularities" be more acceptable? But please do not ask me to change "morally deformed" - this is the lens through which people must judge the consequence of their words. This is my first book in traditional Chinese. Just the thought that students in remote areas of China are learning to speak cause and effect sends shivers in my spine. I hope the govt tolerates this revolution. #Bookofwhy

8.20.19 @2:29am - (Replying to @Chris_Auld) RF isn't useful here because X is not exog. by RF definition. As to precision, DAGs are not very helpful here, with two exceptions: 1. the partial order defined in 2. Ratios of correlation coefficients can be estimated by 2SLS. #Bookofwhy @PHuenermund

8.20.19 @2:13am - (Replying to @Chris_Auld) In Fig.5.10 (Causality p.153) Z is not exogenous, and X is not a valid IV in the traditional econ sense (exclusion is violated). So, I am not sure it can be solved using traditional econ methods, and would be eager to learn otherwise. #Bookofwhy

8.20.19 @1:08am - (Replying to @melvinwevers) I would replace "contaminate" with "stain". But "weed" connotes "undesirable" and "unintended" odd balls. I don't believe my fellow Democrats contemplated this kind of embarrassment, and most of them would rather see it disappear, if it didn't appear as Trump's victory.

8.20.19 @12:54am - (Replying to @thehuntinghouse) Thanks for granting my people right to a homeland, something Rashida can never do. But your use of the word "apartheid" proves how easy it is for decent people to fall victim to deceitful propaganda. Did you really fall for it?

8.20.19 @12:23am - (Replying to @pythiccoder) Thanks for the song. Phil Ochs was my favorite singer in the late 1960's, when I was part of the counterculture revolution. Yes, the fear of getting Blacklisted by Zionophobic big-mouths is what prevents my fellow Democrats from calling out Rashida's racism. Someone has to do it.

8.19.19 @11:42pm - (1/ ) Hating to insult or disappoint any of my followers, I was seriously considering your suggestion to create a new twitter handle. But one word you said made me change my mind: "uninformed". I have been reading, writing and researching the Middle East for the past 83 years. I was
8.19.19 @11:42pm - (2/ ) there when Azam Pasha declared (Oct. 11, 1947) "a war of extermination and momentous massacre" on a nation of refugees of which I was a son. And I was here at UCLA (2014) when BDS's Omar Barghouti re-denied my people right to self determination: A new
8.19.19 @11:42pm - (3/ ) twitter handle will give people of your persuasion the illusion that it is impossible for an "informed" person to disagree with their bubble of self righteousness and that "informed" people must be blind to the genocidal aims of BDS and its spokeswomen Rashida and Ilhan.
8.19.19 @11:42pm - (4/ ) I can't do it. I feel an obligation to truth and to history to let followers in your bubble know that "well informed" people exist who view them as gullible instruments in the service of a racist movement called BDS. Many of my colleagues feel same, but keep silent. I cant.

8.19.19 @8:22pm - (Replying to @StevePittelli) I would like very much to learn from a native English speaker what "time in history" these words harken, and why they seem insulting for some people. I chose them as carefully and as informedly as I could, given what I know about these two ladies stand for. Lifelong learning.

8.19.19 @7:42pm - (Replying to @Jacobb_Douglas @PHuenermund and 2 others) Can you elaborate your "how far" question? Please use several tweets and be pedantic about the references. By 3.2, do you mean Section 7.3.2? I cannot parse: "If Y=y and U=u, then X's PO=x." Please help

8.19.19 @6:49am - (Replying to @JonAMichaels) I honestly thought I was charitable, given that these two ladies have been dehumanizing a whole nation on a daily basis. Do you think their hatred is less than "sickly"? Is it controllable? Is it less dangerous if treated as "healthy"? Truly perplexed.

8.19.19 @4:08am - (Replying to @jmtroos) I tried to be charitable.

8.19.19 @3:32am - (Replying to @mom2phd) I am seriously worried about it. However, given that it can also be used for identifying patients "most in need" of a given treatment, I hope the net benefit to society will be positive. BTW, was the technique you mentioned based on counterfactual bounds? #Bookofwhy

8.19.19 @3:18am - (Replying to @BenWinegard @dabblingfrancis and @clairlemon) Interesting!. Do you have the source? Was I right about the correlation+plausibility combination?

8.19.19 @3:07am - The task of identifying individuals who are "susceptible to persuasion" (or "gullible"), has an enormous range of applications. Ang's slides tell you how it can be done using counterfactual logic. The technical paper, with proofs, is here: #bookofwhy

8.19.19 @1:28am - (1/ ) (Replying to @yudapearl and @Chris_Auld) 1/I did some further reading in Mann and Wald (MW) 1943, and I am fairly convinced now that their motivation was to facilitate identification of structural parameter and, not knowing S. Wright's, nor any other method, they identified the RF and tried to solve for the parameters.
8.19.19 @1:41am - (2/2) (Replying to @yudapearl and @Chris_Auld) This brings us to the question of whether today, that we know many other methods, should RF's be as revered as they were in the past? See what we can do today with other methods, and where RF will fail. Causality page 153. #Bookofwhy

8.19.19 @1:16am - Another new arrival, for those who do not speak Portuguese:

8.19.19 @1:01am - New arrival: My first Portuguese translation in paperback #Bookofwhy = O Livro Do Porque

8.19.19 @12:53am - Omar and Tlaib are already a "national scandal". Two morally deformed weeds in my party that have contaminated the US Congress with their sickly hatred of a certain country, and will continue to embarrass American democracy till someone (their voters?) say: Enough!

8.19.19 @12:31am - (Replying to @dabblingfrancis and @clairlemon) In the case of smoking, to be precise, it was a combination of correlations and "plausibility judgement", which is a type of causal assumptions. #Bookofwhy Chapter 5

8.18.19 @11:23pm - (Replying to @stephensenn @AlexJohnLondon and 3 others) Perhaps you can also tell us what functions were assumed for the arrows of Fig. 6.9(b), before you ran the simulation. How you made sure that the ellipses would be aligned as in Fig. 6.9(a) and, most importantly, what did you expect to learn from the simulation.#Bookofwhy

8.18.19 @8:16pm - A bit of history. The first BDS-style campaign started April 1, 1933, when the Nazi's boycotted Jewish businesses. It followed 3 years later, 1936, by the Palestinians, in their efforts to prevent European Jews from escaping - the most inhumane immigration policy in human history

8.18.19 @4:58pm - (Replying to @AlexJohnLondon @el_hult and 3 others) Don't miss Lord's Paradox, in its unmolested version. Just two plans, A and B, just two statisticians, just an innocent story, no ghosts, no red herrings. #Bookofwhy.

8.18.19 @3:11pm - (Replying to @stephensenn) The quoted passage has suffered the wrath of several misinterpretations, some suggesting a dining Hall serving several diets, and other complications. Fig. 6.9(b) disambiguate the data generation process. Were your two figures generated by this process? #Bookofwhy

8.18.19 @2:44pm - (Replying to @stephensenn @RonKenett and 2 others) If I have not provided explanation it must be that I do not understand those figures or, more specifically, how the data were generated, and whether both were generated in accordance with Fig. 6.9(b) -- the data generating model of Lord's Paradox. #Bookofwhy

8.18.19 @1:53pm - (Replying to @RonKenett @stephensenn and 2 others) Disagree! None of the versions separates Diet from Hall-of-Residence from Dining-Hall. Why complicate things? WHY? As summarized here: Lord Paradox is simple, and the decision between the two analysts is a simple exercise in causal analysis #Bookofwhy

8.18.19 @2:50am - (Replying to @broudsov and @rkarmani) Much of what was debated for centuries should be re-debated in our century, because AI brings to the table the first operationalization of ideas that were debated in the abstract. #Bookofwhy

8.18.19 @1:05am - (Replying to @broudsov and @rkarmani) I feel uncomfortable mixing reality "learning the cause" with fantasy: "rationalization". If you learn it, then it is there. The word "rationalization" weakens the necessity of such an understanding, as if there an alternative way of explaining observed regularities. #Bookofwhy

8.17.19 @10:26pm - (Replying to @heymanitshayden @stephensenn and 2 others) The Primer delves into the mathematics, and has many illustrating examples, thus empowering you to DO causal inference, as opposed to TALK ABOUT it. #Bookofwhy

8.17.19 @9:56pm - (Replying to @rkarman) My take: "Statistics is a torturous workaround until we learn the cause, then it begins to make sense."#Bookofwhy

8.17.19 @9:49pm - (Replying to @stephensenn @MadelynTheRose and @NP_Jewell) To read Primer is an irreversible decision. Like #Bookofwhy, it is going to be painful and, like #Bookofwhy, it is going to be transformative. Let me know when I can tweet our comrades in the trenches: "Stephen Senn has joined the revolution."

8.17.19 @4:44am - (Replying to @EngineerDiet @optempirics and @FatWhiteFamily) The words "mathematically convenient" are misleading when we compare designing digital circuits using Boolean algebra vs. equations of electrons and holes in semiconductors. The difference is between the doable and the undoable, and can only be appreciated by doing #Bookofwhy

8.17.19 @2:06am - I retweet this reply because I get many such inquiries: "Everything you can do with DAGs you can do with ...." The analogy is "Everything you can do with computers you can do with the theory of semiconductors, yet Boolean algebra helps, and so does programming language"#Bookofwhy

8.16.19 @6:26pm - (Replying to @optempirics and @FatWhiteFamily) I confirm your suspicion that everything you can represent in a DAG you can also represent in math equations. The former is an abstraction of the latter. Now, to appreciate the former, take any 3 variables in the latter and check if X is independent of Y given Z. #Bookofwhy

8.16.19 @12:55pm - (Replying to @mattshomepage) A quick scan warns me of something fishy here: all terms are probabilistic, I see no causal model, no causal assumptions, hence - no causal conclusions. Unless I am missing the key, ie. the input model, and then the question arises, why hide it?. #Bookofwhy

8.15.19 @2:57pm - (Replying to @Chris_Auld) Thanks so much Chris. I was about to post a query for the origin of the name. Now we can examine what Mann and Wald had in mind before the concept got distorted. #Bookofwhy.

8.15.19 @3:04am - (Replying to @stephensenn and @RonKenett) What's wrong with assuming " the diet being varied between Halls." or as #Bookofwhy says it: "the students eat in one of two dining halls with different diets". Each hall serves its own diet. What's so "WRONG" in assuming it, and moving forward to the paradox.

8.15.19 @2:34am - This wonderful road map on "good and bad controls" reminds me of a paper I wrote with S. Greenland on "Adjustments and their Consequences" Here the same issues are discussed in epi-vocabulary -- good for our dictionary #Bookofwhy

8.15.19 @2:18am - Replying to @stephensenn and @RonKenett) I am incapable of such offense. First, because terms such as "varying treatment within centers" is not in my vocabulary and, second, because I don't see such variations in Lord's story, nor in the "clean" data generation process of fig. 6.9. #Bookofwhy

8.15.19 @12:03am - (Replying to @stephensenn and @RonKenett) The adjustment equation is this: P(Y|do(Diet)) = SUM W_I P(Y|Diet,WI) P(WI) taken from, and telling us precisely how things are estimated. No weaknesses, no "two cases", no complications -- straight causal analysis and a paradox dissolved. #Bookofwhy

8.14.19 @11:47pm - (Replying to @jdramirezc) I am familiar with the examples cited. But here we are trying to understand the logic behind them. i.e., what information is ADDED when an economist says "This is a REDUCED FORM EQ." Or, What would we miss if we haven't heard him say it? Anything useful? #Bookofwhy

8.14.19 @11:33pm - (Replying to @jdramirezc) The supply-demand example was also in Haile's slides, but he insisted on all arguments being exogeneous. Relaxing this, turns RF into a regression equation - impossible. Our question: What would science miss if, suddenly, economists forget such a concept ever existed? #Bookofwhy

8.14.19 @10:41pm - (Replying to @jdramirezc) Your interpretation of Reduced Form is much broader than anything we heard here from economists, and would fit almost anything. Low and Maghir, likewise, are not providing a definition, they just talk around it. One day, we outsiders will get it too, I am sure. #Bookofwhy

8.14.19 @9:05pm - Continuing our efforts to improve communication with economists, Carlos and Andrew have posted: "A Crash Course in Good and Bad Control" -A road map guide for the perplexed traveler, novice and seasoned. Enjoy! #Bookofwhy

8.14.19 @8:03pm - (Replying to @jdramirezc) Thanks for the reference. We are exploring how economists think, so every such source is valuable. Oh, almost forgot, what does REF mean to you? #Bookofwhy

8.14.19 @6:04pm - (Replying to @Chris_Auld) Great!! So, if RF is useful "because they can be solved on a computer", so can many other equations, especially those that are OLS identifiable. Agree? And you and I (not sure about Imbens) can easily tell who those equations are. #Bookofwhy

8.14.19 @5:59pm - (Replying to @Chris_Auld) I do not view RF as arbitrary. On the contrary, it is very well defined, once you have a structural model. I am still exploring though how economists think about it, and why they chose (historically) to give those sort of equation a special name "REDUCED FORM". #Bookofwhy

8.14.19 @5:54pm - (Replying to @jdramirezc) Not really "simpler". Imagine a system with 100 exogenous variables, in which one structural equation has only three variables. The latter is "simple", "simpler than any reduced form one can write, yet we are fobidden from calling it "reduced form". #Bookofwhy

8.14.19 @5:05pm - (Replying to @stephensenn and @RonKenett) i) 'controlling for the initial weight' is achieved through the adjustment equation in my tweet. ii) fig. 6.9b would look the same if diets had been independently varied at the level of student. 6.9b assumes some students deviate (randomly) from the dictated diet. #Bookofwhy

8.14.19 @4:44pm - (Replying to @Chris_Auld) Are we in agreement then that identification considerations played a role in econometricians deciding which "solution" deserve the RF name and which do not? Else, why are we insisting on "exogeneity" of ALL arguments? #Bookofwhy

8.14.19 @3:57pm - (Replying to @Chris_Auld) This is my point. We have a "solution" to the system of equations and, yet, we DO NOT call this solution "reduced form". Why, because all observed variables in the RFE must be exogenous. I have no opinion of my own, just trying to understand how economists think about RFE.

8.14.19 @2:29pm - (Replying to @Chris_Auld) There are many solutions to the system of equations which were not baptized with a NAME; why RF? E.g., consider the front door: X-->Z--->Y, with X<-U->Y . Y=f(Z,U) is a solution, and so is Y=g(X,U). Behold, no name and no OLS. Further, why do u say "sometimes"? #Bookofwhy

8.14.19 @6:23am - (Replying to @stephensenn and @RonKenett) To reiterate. #Bookofwhy aims only to resolve the paradox under the assumptions stated in Fig. 6.9 (b). Here all assumptions are stated, because it is a data-generating process. Still, the paradox persists in our minds, then resolved., mission accomplished.

8.14.19 @5:38am - (Replying to @JadePinkSameera) Great! Please tell us if you find Lord's paradox to be paradoxical and if #Bookofwhy left anything undone in the way it explains away the paradox.

8.14.19 @5:28am - Foreign Affairs Magazine just published an excellent review of "Possible Minds" where the future of AI is discussed from 25 angles. (link: . My chapter can be viewed here (link: proposing model-based ML to overcome Deep-Learning limitations.

8.14.19 @5:21am - (Replying to @stephensenn and @RonKenett) #Bookofwhy may fail to do many things under the sun, but one thing it does not fail to do -- resolve a clash between the two intuitions, a clash that has baffled many analysts before, and still baffles them today, even under the "clean" data generation conditions of fig. 6.9(b)

8.14.19 @3:43am - (Replying to @RonKenett and @stephensenn) Different tasks do not make differences in conclusions. Accounting for "blocking structure" and "study design" and perhaps "quantum uncertainties" are unrelated the task of resolving a clash between two intuitions that persist EVEN in the "assumed causality links" #Bookofwhy

8.14.19 @2:05am - (Replying to @yudapearl @Chris_Auld and 7 others) Continuing our explorations of "Reduced Form Equations" (RFE) and what they mean to economists, I have tweeted this thread: (link: . I hope RFE experts approve of the way I explain it to my students. #Bookofwhy

8.14.19 @1:55am - (1/ ) Continuing our exploration of "Reduced Form Equations" (RFE) and what they mean to economists, let me address some hard questions that CI analysts frequently ask. Q1: Isn't a RFE just a regression equation? A1. Absolutely Not! A RFE carries causal information, a regression
8.14.19 @1:55am - (2/ ) equation does not. Q2: Isn't a RFE just a structural equation? A1. No! Although a RFE carries causal information (much like a structural equation) the RFE may not appear as such in the structural model; it is derived from many such equations though functional composition.
8.1 .19 @1:55am - (3/ ) (The output-instrument in the IV setting is a typical example). Q3: One may derive many equations from a structural model; what makes a RFE so special to deserve its own name. A3: It is exceptional because it comes with a license of identification by OLS. This is not usually
8.1 .19 @1:55am - (4/ ) the case for other derivable equations, say those relating two endogenous variables. Q4: But some of those other equations ARE identified by OLS; why haven't they been baptized with a name? A4:Because traditionally, economists did not have an easy way of telling which equation
8.14.19 @1:55am - (5/5) enjoys identification by OLS and which does not. Now they do, so it is quite likely that next generation econ. texts will introduce new names. For example, every structural equation identifiable by OLS should be so recognized. #Bookofwhy @PHuenermund @causalinf @analisereal

8.13.19 @11:02pm - (Replying to @EpiEllie) Intrigued by your "hotter take". Are descriptive analysts using the term "adjustment" in lieu of "stratification"? How would you explain to them what the difference is? #Bookofwhy

8.13.19 @10:45pm - (Replying to @RonKenett and @stephensenn) Why assume that there are "differences" between us? I am trying to explain a clash between two strongly held intuitions (no building systems, no automation) and Senn is trying to do something else (no intuitions no causal assumptions). I do not see "differences" #Bookofwhy

8.13.19 @9:33pm - (Replying to @Prof_Livengood and @OUPPhilosophy) John to painter: "Paint the wall either green or purple". Painter to John: "Your wife will get pretty angry." John was imperative, Painter was indicative. Both used a disjunctive do-operator, as interpreted here: (link: #Bookofwhy

8.13.19 @5:05pm - (Replying to @Prof_Livengood and @OUPPhilosophy) I will be happy to have a long conversation and to learn what the distinction is between "imperative" vs. 'indicative" in the context of the do-operator. Do they make different claims? #Bookofwhy

8.13.19 @3:53pm - My latest offering on "Lord's Paradox and the Power of Causal Thinking" with a few goodies for the curious and empowered (link: #Bookofwhy

8.13.19 @8:48am - (Replying to @NikosTzagarakis and @intoolab) Glad another person beside me sees how important vocabulary is to thinking straight.

8.13.19 @8:39am - (Replying to @BruceTedesco) There is something addictive in the word "Baysianism" (as in the word "induction"), perhaps because it seems to capture so many cognitive functions that blinds out from seeing what it can't. #Bookofwhy

8.13.19 @2:31am - (Replying to @imleslahdin and @spyrosmakrid) I commented on this paper a few days ago, and so did other ML folks.

8.13.19 @2:05am - Very interesting post. I wonder what it would take for a philosopher to read (link: and change the title of Section 2 to read: "What a Great Many Phenomena Bayesian Decision Theory CANNOT Model" #Bookofwhy @OUPPhilosophy

8.12.19 @4:06pm - (Replying to @AndrewPGrieve) When I see a quote I dont remember, I begin to realize how old I must be. 105 !

8.12.19 @1:39pm - (Replying to @yudapearl @Chris_Auld and 7 others) Do experts on "reduced-form" approve of the licenses I am attributing to Y=f(X,Z,U) ? If so, we can continue with what f says about the world, or perhaps with more licenses. #Bookofwhy

8.11.19 @9:46pm - (Replying to @analisereal @autoregress and 7 others) I bet there is some added value to saying "reduced forms", perhaps a pedagogical value, but, since I've never used it, its up to those who do to explain: "what is added?" Perhaps it is just a verbal assurance that someone already taken care of exogeneity. Hard to guess #Bookofwhy

8.11.19 @9:07pm - (Replying to @Chris_Auld @autoregress and 6 others) Sorry, I thought you said that (quoting): "equation 3.46 in Causality is a ratio of two reduced forms", so I looked at 3.46 and found there a ratio of two cond. expectations. I will try to be less pedantic. (Unless you would you like me to be more pedantic?) #Bookofwhy

8.11.19 @8:55pm - (Replying to @yudapearl @Chris_Auld and 7 others) 2/ infer: E[Y|do(X)] = E[Y|X], E[Y|do(Z)] = E[Y|Z], and E[Y|do(X),do(Z)] = E[Y|X,Z], regardless of the form of f. License #2. Y=f(X,Z,U) is RFE then so is W=g(X,Z,U) for some g, where W is ANY variable not in {X,Z}. Hence, E[W|do(X)] = E[W|X]. Will continue if allowed. #Bookofwhy

8.11.19 @8:39pm - (Replying to @yudapearl @Chris_Auld and 7 others) 1/ What is a Reduced Form Equation (RFE) - a layman interpretation. Introduction: an RFE is a causal, not statistical statement. It makes causal claims about the world, and provides licenses for causal inference. License #1- If Y=f(X,Z,U) is RFE, then thou have the license to

8.11.19 @8:13pm - (Replying to @Chris_Auld @autoregress and 6 others) Sorry, but Eq. 3.46 in Causality reads: b = E[Y|z]/E[X|z] It is a ratio of two conditional expectations, not "two reduced forms". The former are descriptive, the latters are causal -- like water and oil. IOW, E[Y|x] conveys no causal information, RF's do! Can't equate.#Bookofwhy

8.11.19 @7:19pm - (1/ ) (Replying to @Chris_Auld @autoregress and 6 others) I am the last one to claim that a concept is not "useful" its usefulness however needs to be clear, else it turns into "abusefulness." I am happy to finally hear from someone what it means to that someone, not to another person, or to some textbook, and I will try to refine it,
8.11.19 @7:33pm - (Replying to @yudapearl @Chris_Auld and 7 others) so as to explain it to my students, for whom "meaning" comes either as a claim about the world, or as a license to conclude something about the world. I will try translating your "meaning" of RF to their language, starting with the licences. Pls check for promiscuity #Bookofwhy.

8.11.19 @6:01pm - (Replying to @Chris_Auld @autoregress and 6 others) So, @Chris_Auld perhaps you can tell us in plain language what the sentence "Y=f(X,Z,U) is a reduced form equation" means to YOU, so I can tell my students how to communicate with economists. Let's forget MHE and ancient history. What does the label RF add? #Bookofwhy

8.11.19 @5:45pm - (Replying to @steventberry @autoregress and 5 others) So, if we agree that economists today are not clear on what is meant by RF equation, perhaps we should give them a few hints on how to explain RF when communicating with DAG-minded folks. Right now, they dont really know what the explanation should entail. Ready? #Bookofwhy

8.11.19 @3:52pm - (Replying to @steventberry @MariaGlymour and 5 others) We shouldn't argue about ancient history. I am trying to inform my students and other DAG-minded researchers how to communicate with economists, i.e., when an economist tells you "This is a reduced form equation" what does he/she mean? What does the equation claim etc.#Bookofwhy

8.11.19 @3:05pm - (Replying to @omaclaren @fuzzydunlop123 and 7 others) When my grandson insisted on understanding what makes a water molecule accelerate when it goes through a constriction, I said: its neighbors from behind are pushing it stronger than the neighbors in front, they are closer, hence more pressing. #Bookofwhy

8.11.19 @2:33pm - (Replying to @lewbel) I am not aware of the notions of "incompleteness" or "incoherence". Is there a simple example for the uninitiated?

8.11.19 @2:12pm - (Replying to @onnlucky @dchackethal and 2 others) I am convinced that, if Popper was alive, he would buy the Ladder of Causation from #Bookofwhy, after realizing that "induction" is too general a term to be operationalized.

8.11.19 @3:36am - (1/3) Econ. readers asked if they can get hold of those magical night-vision goggles that tell us which causal effects in an econ. model are identifiable by OLS (and how). The answer is embarrassingly simple: Consider Model 2 in p. 163 of (link:
8.11.19 @3:36am - (2/3) Take any two variables, say Z_3 and Y. If you can find a set S of observed variables (e.g., S={W_1,W_2}), non-descendants of Z_3, that block all back doors paths from Z_3 to Y, you are done; the coefficient a in the regression Y = a*Z_3 + b1*W_1 +b2*W_2 gives you the right
8.11.19 @3:36am - (3/3) answer. A similar goggle works for a single structural parameter (See page 84 of Primer (link: Duck soup. This re-begs the question whether the restricted notion of "reduced form" is still needed in 2019. @EconBookClub @lewbel #Bookofwhy.

8.11.19 @2:25am - (Replying to @mnjp) Glad you took the firing squad seriously. Some people think it is just a "toy problem" and never learn what counterfactuals mean. #Bookofwhy

8.11.19 @2:20am - (Replying to @Jabaluck @fuzzydunlop123 and 6 others) Adding my interpretation of Haile's interpretation of "reduced form". I hope I am right, or at least faithful:

8.10.19 @6:09pm - (Replying to @omaclaren @fuzzydunlop123 and 7 others) You are confirming my observation that people tend to cite the name of the conservation law as an explanation of observed phenomena. Yet naive me always wanted to know: why would a fluid particle accelerate when getting into the narrow part of the tube. Now I know why. #Bookofwhy

8.10.19 @5:25pm - (Replying to @omaclaren @fuzzydunlop123 and 7 others) How would Dowe "explain" Venturi's effect (pressure measured in a fluid flowing in a pipe of varying cross section).?

8.10.19 @5:07pm - (Replying to @fuzzydunlop123 @omaclaren and 7 others) Conservation laws are non-causal emergent properties of dynamic systems. The latters are causal, because each particle is "responding" to the forces in its neighborhood. Interestingly, people tend to accept the name of the conservation law as an explanation of events #Bookofwhy

8.10.19 @4:39pm - (1/ ) (Replying to @MariaGlymour @Jabaluck and 5 others) 1/ The translation of Phil's definition to DAG language is simple: Y=f(X,Z,U) is "reduced form" iff X, Z and U are ALL the root nodes that are ancestors of Y, both observed and unobserved. The question remains why would an analyst write down such an equation when all its claims
8.10.19 @4:44pm - (2/ ) (Replying to @yudapearl @MariaGlymour and 6 others) are obvious from the graph? For example, the claim that the causal effect of X (and Z) on Y can be estimated by OLS. The answer I believe has to do with the fact that the notion of "reduced form" emerged in a period when economists where desperate for conditions to justify
8.10.19 @4:51pm - (3/3) (Replying to @yudapearl @MariaGlymour and 6 others) identification by OLS. Today they are fortunate to have night-vision goggles that tell them immediately which effects are estimable by OLS, and which regressors to include/exclude (link: This was not always the case in the history of economics. #Bookofwhy

8.10.19 @6:48am - (Replying to @mathtick) I dont recall commenting on it. Will add to my "to do" pile.

8.10.19 @6:44am - (Replying to @tarinziyaee) No changes except for the many errata shown here: (link: (link: (link: and marked in red. The new edition will incorporate them in a clean text.

8.10.19 @2:13am - (Replying to @RonKenett and @ShalitUri) I do not recommend going in Hastie's direction (no pun) of starting with black-box and then asking: when would some statistical estimate have a "causal interpretation". #Bookofwhy goes the other way: start with what you know and ask if you can estimate what you want to know.

8.10.19 @2:07am - Correcting a link in a previous post. The new errata for the Causal Inference Primer book can be accessed here: (link: (marked in red). #Bookofwhy

8.9.19 @10:57pm - (Replying to @fuzzydunlop123 @lewbel and 6 others) Here we are, trying to out-guess each other what Haile meant by "reduced form", and MHE, and so and so... Something is terribly wrong in 2019 science if we can't find a red-blooded economist willing to take a stand and say: TO ME, "reduced form" means... How about it? #Bookofwhy

8.9.19 @10:37pm - (Replying to @fuzzydunlop123 @IvanWerning and 6 others) I agree here, "model" cannot replace "structural model," because statisticians think that the assumption of normality is a "model" and some MHE students think that a "regression equation" is a "model". The word "structural" tells us what assumptions it carries. #Bookofwhy

8.9.19 @10:18pm - (Replying to @lewbel @steventberry and 5 others) I have no doubt that "reduced form" exists or can be concocted. But the question is: What useful information does it provide to @lewbel (not MHE). Put differently, what would science miss if, suddenly, all economists get amnesia and forget such a concept ever existed? #Bookofwhy

8.9.19 @9:29pm - And while we are speaking of Causal Inference - Primer, (link:, good news comes from Wiley: They are preparing a revised edition to hit the shelves soon. It will include new errata collected and composed by Scott Meuller. See (link: #Bookofwhy

8.9.19 @8:19pm - (Replying to @thanhnguyentang) You can start with #Bookofwhy for fun,history and philosophy, but if you want to delve straight to the mathematics, and still have fun, there is no better introduction than Primer (link:

8.9.19 @8:01pm - (Replying to @IvanWerning @Jabaluck and 5 others) @IvanWening, the more I read your Tweets the more I agree with you (r u sure u r an economist?). The rearrangements you are permitting are permissible as long as they preserve information. Algebraic rearrangements are NOT information preserving. Nor are "Reduced forms" #Bookofwhy

8.9.19 @7:41pm - (Replying to @lewbel @steventberry and 5 others) "Reduced form" vs. "Confused form": Can anyone define what it means to him/her (not to Phil or MHE) and why it should not be purged from econometric discourse w/o loss of information? #Bookofwhy

8.9.19 @7:33pm - (Replying to @lewbel @steventberry and 5 others) I must be missing something important. If the "reduced form" is descriptive, how can it impose untestable assumptions? More basic: Does the "reduce form" preserve the structural assumptions carried by the structural model of which it is a "reduced form"? #Bookofwhy

8.9.19 @7:16pm - (Replying to @IvanWerning @Jabaluck and 5 others) Totally agree that "reduced form" is a source of much confusion. It gives the impression that it is merely a syntactic transformation of "form" with no loss of information. But if it is "descriptive", then we loose all the causal information that SEM provides. Shun! #Bookofwhy

8.9.19 @4:55pm - Machine Learning enthusiasts will be interested in George Lawton's new post titled "Causal Deep Learning Teaches AI to ask why" (link: I am not familiar with all the actors mentioned in the story, but I am glad ML is moving beyond curve fitting #Bookofwhy

8.9.19 @6:19am - (Replying to @Jacobb_Douglas @maximananyev and 6 others) The draft lottery, the price of beans in China, the court decision (see @Bookofwhy Chapter 1), etc. its a variable that only sick/creative stretch of imagination would deem it relevant to the context of interest.

8.9.19 @4:39am - (Replying to @sarahmrose @SHamiltonian and 2 others) As I said in my confession (link: I spend time on such debates knowing that there are dozens of silent and bright students out there, listening to the conversation, and gathering ammunition for future defense of commonsense. #Bookofwhy

8.9.19 @1:14am - (Replying to @fuzzydunlop123 @maximananyev and 6 others) One should add though that this is not done with the intention to "fool." When you don't SEE your assumptions you tend to believe that your "experiment" takes care of them. The Talmud says: If our eye were given the power to see our assumptions, we won't get out of bed #Bookofwhy

8.9.19 @1:04am - (Replying to @paulgp @maximananyev and 4 others) But, in addition to saving those answers from oblivion, the DAG also permits you to COMBINE them, and thus answer questions that DAG-averse folks CAN'T. E.g.,"Is the treatment ignorable?" or "Are there any testable implications". That's why these folks rarely ask them. #Bookofwhy

8.9.19 @12:10am - (Replying to @maximananyev @Jabaluck and 5 others) A DAG is none other but a collection of answers to the question: "What is the source of variation in variable X?" recorded as an arrow into X. What is often called "The DAG Approach" is consulting those answers, instead of re-asking when they're needed in estimation. #Bookofwhy

8.8.19 @11:44pm - A question on Quora read: ML is becoming too competitive! Should a person wishing to become a ML researcher give up? As an aspiring ML researcher, I had to tell them the truth:

8.8.19 @7:54pm - (Replying to @DonskerClass @Jabaluck and 5 others) This is one way of handling potential misspecifications, (good paper) another is sensitivity analysis (if done correctly) which passes the burden on to a Plausibility judgment on the likelihood that certain edges can attain a certain strength. #Bookofwhy

8.8.19 @6:54pm - (Replying to @causalinf @steventberry and 3 others) Lewbel's taxonomy notwithstanding, "Design" cannot replace "identification". "Design" connotes an option an analyst may or may not choose, "identification" is a property of your model; a causal query is either identifiable or not (given the model) no matter what you do.#Bookofwhy

8.8.19 @4:41pm - (Replying to @causalinf @steventberry and 2 others) @causalinf, Are you really going to replace "identification" with "design"???. Clarity with escapism?? "Identification" is perhaps the clearest notion developed by economists --"Design" is the foggiest . @Undercoverhist, as historian, watch the last days of clarity.#Bookofwhy

8.8.19 @4:17pm - (Replying to @fuzzydunlop123 @Jabaluck and 5 others) If someone succeeded in eliminating the word "design" from the literature (perhaps leave it in RCT contexts) clarity will shine brightly on causality-land. A catch-all for "I wish to be more specific". I hope it's avoided in #Bookofwhy. Just checked, it's used mostly harmlessly.

8.8.19 @2:38pm - (Replying to @Jabaluck @Jacobb_Douglas and 2 others) Please do not put words in my mouth. Conditional ignorabitlity assumptions are inscrutable because they are far removed from what we know, as proven by (1) PO textbooks and (2) PO folks refusing the test: Given a story (no DAG), tell me if "Y_x is ind. of X given Z" #Bookofwhy

8.8.19 @2:24pm - (Replying to @fuzzydunlop123 @Jabaluck and 6 others) I can't tell if the Yale folks stand behind the slides, nor can I tell whether the slides reflect outdated terminology or advocated terminology. Its healthy to see them going through some sort of soul-searching catharsis. Let's summarize what we learned by email

8.8.19 @1:55pm - (Replying to @fuzzydunlop123 @Jabaluck and 5 others) @fuzzydunlop123, I cant follow the barrage of tweets in the wake of Phil Haile slides, but you make consistent sense. Bringing up (link: (link: was timely -- we haven't seen a drastic improvement in Econ. texts yet. Is it coming? #Bookofwhy

8.8.19 @11:42am - (Replying to @steventberry @Jabaluck and 4 others) Very interesting and helpful. Thanks for posting. Is this set of slides representative of the current thinking at Yale-Economics ? Or are they still undergoing internal debate regarding the precise meanings of terms? #Bookofwhy

8.8.19 @6:05am - (Replying to @georgemsavva @stephensenn and @statsepi) But if Hall is perfectly correlated with Diet, why won't the effect of Diet on Gain not be the same as the effect of Hall on Gain. Once we decide what effect we need, the diagram delivers it for you, since we do not have unobserved confounders in the model. #Bookofwhy

8.8.19 @5:37am - (Replying to @_amirrahnama) I agree with your general statement above, though I have not read the paper by @tmiller_unimelb . I would be delighted to find out that causality and explanation are well understood in AI. #Bookofwhy

8.8.19 @3:01am - (Replying to @georgemsavva @stephensenn and @statsepi @georgemsavva, you hit it on the nail, thanks for making it explicit, and for stressing that #Bookofwhy deals with Lord's paradox, not with experimental design. Moreover, if we suspect Hall and Diet have separate effects on Weight, another diagram would resolve it just as well.

8.8.19 @2:39am - For those concerned with issues of Free Press and East-West relations, a Panel on Aug 15 in LA would be most illuminating. See (link: RSVP required.

8.8.19 @2:17am - (Replying to @stephensenn and @statsepi) You have to trust my honesty if I say: I don't have the slightest idea how this is related to Lord's dilemma with TWO HALLS, each serving ONE diet, and students are SHOWN what diet is served in each. Please start here, and tell us what factor is neglected in #Bookofwhy

8.8.19 @2:08am - (Replying to @statsepi and @stephensenn) Great news indeed. Humble advice: Try to focus on why YOU (not Nedler) are surprised by Lord's dilemma, and see for yourself whether #Bookofwhy does not pacify your surprise to the point where no such story can ever surprise you again.

8.8.19 @2:01am - (Replying to @stephensenn and @RonKenett) "reasonable under common circumstances " is insufficient. Resolution comes from understanding why two analyses, which were reasonable, even compelling under "common circumstances" suddenly cease to be reasonable under the current circumstances. Watch it explicated in #Bookofwhy

8.8.19 @1:50am - (Replying to @zaffama) You just gave a clear example. In nature, the sun's motion determines its angle, generating sun rays that are reflected from the atmosphere, sensed by the rooster, and make him crow. The sun's motion also determine the sun's angle next hour, which we call "Sunrise". #Bookofwhy

8.8.19 @1:35am - (Replying to @zaffama) My point was to show that the direction of causal effect can be opposite to the times of which events are observed or reported, but (God forbids!) not opposite to time's in the data-generating process. #Bookofwhy

8.8.19 @1:28am - (Replying to @yudapearl @RonKenett and @stephensenn) To be consistent with my request, I believe I did an honest job in confessing my "surprises" in my 3rd comment on Senn's post. Here: (link: . And note my conclusion: Yes, causal analysis does dissolve the clash of intuitions in Lord's paradox. #Bookofwhy

8.8.19 @1:11am - (Replying to @RonKenett and @stephensenn) Another comment: I find it hard to discuss this topic without knowing where my discussant stands on the issue of "surprise", or whether he/she is at all surprised by Lord's story. So, I beg you, and future discussants to start with an "analysis of surprise" #Bookofwhy

8.8.19 @12:58am - (Replying to @RonKenett and @stephensenn) Never mind engineering challenges and robotics. The #Bookofwhy chapter on Lord's paradox is an exercise in scientific psychology: Why are scientists surprised and vexed by the story? Can causal logic dissolve this surprise? The rest is irrelevant to our discussion. #Bookofwhy

8.8.19 @12:45am - (Replying to @DanielNevo @Jacobb_Douglas and @PHuenermund) The differences between the (in)dependencies of Y(a) and Y(not-a) do not show in the structure of the SWIG, this information comes from outside the graphical model. #Bookofwhy

8.8.19 @12:38am - (Replying to @PHuenermund @MariaGlymour and @Jacobb_Douglas) OK, Yielding to Maria and Paul. But please try to make the context clear to an engineer (like me), who can only think in terms of (1) what we know, (2) what we want to know and (3)what data we have available. Input-Output. #Bookofwhy

8.7.19 @11:18pm - (Replying to @DanielNevo @Jacobb_Douglas and @PHuenermund) Agree on "alleviated". But you do not need SWIGs for that task, an ordinary DAG can tell you all about d-separation and back-door tells you all about ignorability and, if you really need to, it shows it to you explicitly, see Causality p.343, (link:, #Bookofwhy

8.7.19 @7:06pm - (Replying to @Jacobb_Douglas and @PHuenermund) Best place to ask is by email. A "typical case" for PO is "we assume, as usual, that D is conditionally ignorable given X". A "typical case" in science is: "Given a story on 3-4 variables, lets see if measuring X would help us get the effect of D on Y . #Bookofwhy

8.7.19 @4:53pm - (Replying to @roieki and @SamHarrisOrg) Eize Perek?

8.7.19 @2:01pm - (1/2) (Replying to @PHuenermund and @Jacobb_Douglas) That is why the answer to the question "Can it be done in PO ?" is plain NO. If we say YES, the impression is created that it is just a matter of a few more computations. It is hard for people to appreciate the difference between tractable and intractable. And that is why PO
8.7.19 @2:07pm - (2/2) (Replying to @yudapearl @PHuenermund and @Jacobb_Douglas) PO folks dread toy problems, where the intractability shows immediately. It is like "solving" a polynomial assuming that someone else already computed its roots. #Bookofwhy

8.7.19 @4:38am - (1/4) I would like to welcome the 500 new followers who have joined us on Tweeter since @SamHarrisOrg posted our conversation on his podcast. Welcome to the wonderful land of WHY and, please, be aware of what you are getting yourself into. Our main theme is the #Bookofwhy and
8.7.19 @4:38am - (2/4) the way it attempts to democratize the science of cause and effect and apply it in artificial intelligence, philosophy, and the health and social sciences, see (link: We alert each other to new advances in causal reasoning and new methods of answering causal
8.7.19 @4:38am - (3/4) questions when all we have are data, assumptions and the logic of causation. We also debate detractors and nitpickers who mistrust fire descending from Mt. Olympus for use by ordinary mortals. I spend time on such debates knowing that for every nitpicker there are dozens of
8.7.19 @4:38am - (4/4) silent and bright students out there, listening to the conversation, and gathering ammunition for future defense of commonsense.
Overall, I hope you find this forum entertaining, challenging, and idea driven.

8.6.19 @10:49pm - Thank you @SamHarrisOrg for having me on your podcast and for our lovely discussion on cause and effect, counterfactuals, free will, and the future of AI. I believe it was your podcast that caused the # of followers on this Tweeter to cross the 20K mark. They'r welcome!#Bookofwhy

8.6.19 @5:13am - (Replying to @stephensenn) The answer is simple: The #Bookofwhy produces a consistent answer to problems defined in the #Bookofwhy, not to problems defined elsewhere, involving several dining halls, or dining halls shifting their diets, or other variations. One thing at a time. The idea of "surgery estimators" is ingenious, it would not occur to me that you can get extra mileage on top of "pruned estimators". However, is Figure 2(b) the simplest example to show this extra mileage?? Would c-equivalence help here? (link: #Bookofwhy

8.5.19 @11:18pm - (Replying to @jaketapper and @RashidaTlaib) @Jaketapper is 100% right, Congresswoman @RashidaTlaib lies. She knows that Palestinians want one more thing, in addition to posing as human right advocates -- dismantling their neighbors. If I am wrong, let us say so publically. She can't!

8.5.19 @7:14am - (Replying to @doinkboy @GRich_Cinci and 2 others) This is what causal inference is all about: "interpret it causal, given a set of assumptions (i.e., a causal model)." Except that the "interpretation" is no longer whimsical, as it used to be, it must obey the logic of causation. And this is what makes it "causal" #Bookofwhy Their numbers is still rising, I hope one of you continues --an amazing phenomenon.

8.5.19 @4:16am - I have just posted my 3rd comment on Lord Paradox (link: Here, I returned to my original goal of empowering readers with an understanding of the origin of the paradox and what we can learn from it. Red herrings have been taking too much of our time. #Bookofwhy
8.5.19 @4:16am - It is important to add that the huge literature on Simpson's and Lord's paradoxes attests to a century of scientific frustration with a simple causal problem, deeply entrenched in out intuition, yet helplessly begging for a formal language to get resolved. I can count dozens of
8.5.19 @4:16am - red herrings thrown at this stubborn problem, to deflect attention from its obvious resolution. Why? Because the latter requires the acceptance of a new language - the hardest transition for adults. I am grateful for the opportunity to talk to thousands of young followers here
8.5.19 @4:16am - on Twitter and convey to them my honest conviction: Chapter 6 of #Bookofwhy (paradoxes galore) is a condense summary of a century of confusion, and a powerful recipe for deliverance from that confusion. Additional references are: (link: (Lord's paradox),
8.5.19 @4:16am - (Simpson's paradox) (link: (Sure-thing Principle) And Chapter 6 of Causality (link: , where I document dozens of red-herrings 1900-2000.

8.5.19 @1:16am - Replying to @mendel_random and @oacarah) The point is: "discovered using a causal model, later depicted as diagrams". Red herrings do not stop us from discussing an issue. OK. We get your point: DAGs dont handle cycles and calculus does't handle non-differentiable functions. Can we get on with the discussion? #Bookofwhy

8.4.19 @7:22pm - (Replying to @mendel_random) I dont think it was worked out in folks' heads too well, they had to use some mathematics, and the mathematics they used came as "equations" which, at the time looked like algebraic but, in time, came to be recognized as non-algebraic, asymmetric, or "structural", #Bookofwhy

8.4.19 @3:13pm - (Replying to @stephensenn) Oh, @stephensenn, you never told us what's surprising you in Lord's story. In other words, must we go to finite sample before you can describe to us the reason for your surprise? #Bookofwhy

8.4.19 @3:09pm - (Replying to @stephensenn) Not wrong. Just irrelevant to Lord's paradox where we assume (using Wainer's model) that students are allowed to choose their own Hall and that each Hall serves ONE diet. If I am surprised with this simple story, I'll try to resolve it HERE, before complicating it. #Bookofwhy

8.4.19 @2:25pm - (Replying to @stephensenn) We differ here. The notion of "probability distribution function" is not meaningless is statistics. Most stat texts start with distributions, then go to "samples from distributions". Lord introduced his paradox at that level, and managed to surprise us; why quit? #Bookofwhy

8.4.19 @2:09pm - Now, speaking specifically about Lord's paradox, the paradox was introduced to us in "asymptotic" terms (ie, using distributions, not samples) and we were surprised. Is it likely that we can resolve our surprise by going to finite samples? or to "block design"? #Bookofwhy

8.4.19 @1:54pm - That is why I am begging folks: "Please, do not tell me 'I am not entirely satisfied' before you tell me why you are surprised (by the paradox) ". I am proud that #Bookofwhy addresses this question (of "surprise") head on, before offering "a resolution".

8.4.19 @1:36pm - (Replying to @mendel_random) No big deal. Replace "using Wright's DAGs" with "using Wright's equations, later depicted as DAGs" and the rest of the Tweet follows, especially "using what you know". The Tweeter discussion was about "using a model", which some folks shun. #Bookofwhy

8.4.19 @1:09pm - Any discussion of "paradoxes" is really an exercise in psychology. Yet we, quantitative analysts, are trying to avoid psychology at all cost. We can't. We must explicate why two strong intuitions seem to clash, and the conditions under which our intuitions fail. See #Bookofwhy

8.4.19 @6:06am - (Replying to @f2harrell @stephensenn and @Lester_Domes) I will expand, in a day or two. But it would help if you reconstruct Lord's paradox in your own way and pin point: What was paradoxical in the story? What was surprising there that deserved the word "paradox"? #Bookofwhy

8.3.19 @7:06pm - (Replying to @EpiEllie) I tried to look into it, but I am missing your research question, ie, the query.

8.3.19 @6:43pm - Good news for missing-data analysts. Karthika Mohan @karthica is joining the Editorial Board of Journal of Causal Inference jttps:// It's a welcoming invitation for articles on modern ways of recovering what we thought to be missing. #Bookofwhy

8.3.19 @4:42pm - I've just posted a comment (link: on S. Senn's n-th attempt to deconstruct Lord's paradox. It ends: "I hope we can now enjoy the power of causal analysis to resolve a paradox that generations of statisticians have found intriguing, if not vexing." #Bookofwhy

8.2.19 @9:25pm - Coming from my fellow statisticians it reminds me of King Solomon's saying: "Let a stranger appraise your work, not your mouth"(Proverbs 27:2). And the Mishna saying: "The baker does not judge his own bread" (Tosefta Yom Tov 3:7). I hope my bread tastes well to others. #Bookofwhy

8.2.19 @9:25pm - Many thanks! Carlos Cinelli came back from JSM-2019 and brought me a gift that I would cherish dearly:

8.2.19 @8:03pm - (Replying to @jasonintrator @glogauer_jakob and @TheIHRA) A group HAS a self. What makes an individual have self is memory, telling him: your experience yesterday informs your decision today. Same with groups, except here the pertinent experience spans centuries, and will easily get lost unless members attain self-determination

8.2.19 @7:26pm - (Replying to @jos_b_mahoney @jasonintrator and @glogauer_jakob) Unlike Jason, I would feel very very lonely, to know that I do not have Canaanite role models who spoke my language, who left my kids beautiful legends and who encoded their experience in a culture baked over 3,000 years. Woefully lonesome!

8.2.19 @5:15pm - (Replying to @jos_b_mahoney @jasonintrator and @glogauer_jakob) Charging people with anti-semitism is stone-age. Edward Said is guilty of a worse offense: Zionophobia, denying Jews the right to define their collective identity after writing volumes on "Orientalism"-the right of Arabs to define themselves. I'm sensitive to logical consistency.

8.2.19 @5:05pm - (Replying to @jasonintrator and @glogauer_jakob) The HE in that prayer (Birkat Hamazon) is God himself, called Ha'Rachaman (the merciful one). True, it was not about "now", but it reflects the continuous aspiration of the Jewish people to eventually regain sovereignty in their historical birth place, i.e., Zionism

8.2.19 @4:54pm - (Replying to @jasonintrator and @glogauer_jakob) Universal liberalism is central indeed to Jewish life. But how: "Thou shall not oppress the stranger, because YOU know what it means, YOU were once a stranger..." Namely, if you forget YOUR collective memory, gone is your universal liberalism. Can't have one without the other.

8.2.19 @4:44pm - (Replying to @jasonintrator and @glogauer_jakob) Their rejectionism started in 1920's, there was no loss of homes or political rights. Not even FEAR of losing homes or rights, as is documented in the Arab newspaper Carmel See "Early Zionists and Arabs"

8.2.19 @4:36pm - (Replying to @jasonintrator and @glogauer_jakob) Herzl, Jews in the diaspora have been divided about the role of Israel as a practical solution to the immediate problems Jews faced then. That role was never debated in my (and your?) grandfather house, where they prayed 3 times a day: "He will walk us in sovereignty to our land"

8.2.19 @4:27pm - (Replying to @jasonintrator and @glogauer_jakob) Your friends from JVP are dangerous indeed, not because they wish us death, but because they recklessly blind themselves to our death, ostensibly in pursuit "universal liberalism." I call them "Jews of Discomfort" here

8.2.19 @4:00pm - (Replying to @jasonintrator and @glogauer_jakob) Yes, I work on this. What is it that I highlighted "over" other facts? Isn't Palestinian rejectionism a fact that cannot be "over highlighted" when it comes to prospects for peace, lifting the occupation, and almost every issue youngsters care about. Do you talk to them about it?

8.2.19 @3:49pm - (Replying to @jasonintrator and @glogauer_jakob) I would never accuse JVP of being motivated by antisemitism; they are motivated by more dangerous forces which I have outlined here:"Our New Maranos"

8.2.19 @2:54pm - (Replying to @jasonintrator and @glogauer_jakob) What is it about the way I represent Palestinians that you think is (1) not factual and (2) not effective with younger American Jews? I hope you are not suggesting young people are not moved by facts? These youngsters were: (link: What about those you meet?

8.2.19 @1:45pm - (Replying to @glogauer_jakob) Truth has its secret way of prevailing, despite BDS's key slogan: "if you repeat a lie long enough, people will fall for it."

8.2.19 @1:32pm - (Replying to @AdanZBecerra1) Agree. And my reason: If we introduce regression before DAGs, students are likely to get trapped in the "regressional confusion" of the 20th century, unable to distinguish structural from regression equations. (link: #Bookofwhy

8.2.19 @5:29am - (Replying to @GivingTools and @causalinf) In my efforts to make causal diagrams palatable to economists I am trying build as much as possible on identification strategies devised by empirical economists. Can you point us to an author or two who came close to outlining diagramatic ideas informally? Thanks. #Bookofwhy

8.1.19 @9:13pm - (1/ ) I am recommending this paper to every data analyst educated by traditional textbooks, which start with regression equations, add and delete regressors, estimate and compare coefficients before and after deletion, and then ask which coefficient has "causal interpretation"
8.1.19 @9:13pm - (2/ ) I was shocked to realize that the majority of data analysts today are products of this culture, trapped in endless confusion, with little chance to snap out of it, since journal editors, reviewers and hiring committees are trapped in the same culture. The new PO framework does
8.1.19 @9:13pm - (3/ ) offer a theoretical escape route from this culture, through the assumption of "conditional ignorability" but, since it is congnitively formidable, practicing analysts must rely on regression arguments. Keele et al examine a causal model (Fig.2) and ask: suppose we regress Y
8.1.19 @9:13pm - (4/ ) on all observed variables; which of the coefficients has any causal interpretation. I have alerted economists to such questions here (link: (3.2.7) so, we can assume they have mastered the techniques by now. Students of #Bookofwhy seeking a gentle way to
8.1.19 @9:13pm - (5/5) approach their mentors or professors or their peers, this is a great channel to motivate them.

8.1.19 @8:35pm - (Replying to @VenkatNagaswamy) I was over-intoxicated by scientific poetry

8.1.19 @8:34pm - (Replying to @yudapearl @Jabaluck and @eliasbareinboim) But, lets deal with what they CAN DO. Let's ask their students to examine Fig. 2 of Keel etal (link: (link: and decide which regression coefficient coincides with a structural parameter. I challenged them here (link: #Bookofwhy

8.1.19 @8:12pm - (Replying to @Jabaluck and @eliasbareinboim) I happened to go over MHE last night. Read: "we follow convention and refer to the difference between the included coefficients in a long regression and a short regression as being determined by the OVB formula (p.44). You say: "They think you should write down a model and then
8.1.19 @8:23pm - (Replying to @yudapearl @Jabaluck and @eliasbareinboim) figure out an estimation strategy ..." I have not seen "a model" written down of what one believe about the world. Even Eq. (3.2.8), which is supposed to be a structural equation, is written after CIA is assumed (conditional ignorability) which we know is cognitively impossible.

8.1.19 @7:58pm - Too bad they cut off my song, just before the tenor!! Not too late. If any reader of #Bookofwhy has ANY unresolved question on DAGs, please check if that question is not answered here (link: If still unresolved, I am here on Tweeter, with an army of resolvers.

8.1.19 @7:48pm - Extremely inviting. "With a book of verse, and thou, beside me, singing in the wilderness, and wilderness is paradise anow." #Bookofwhy

7.31.19 @4:53pm - (1/2) As I was re-reading "Wagged by the DAG" (link: I've found a few collectibles that are relevant to our other discussions: (1) "It would be a disaster if models were allowed to produce information unintended by the modeler." (2) Tools ....
7.31.19 @4:53pm - (2/2) (2) "Tools that are indispensable in solving simple problems are unlikely to become dispensable when problems become more complex." (3) "[In some unnamed cultures,] selecting covariates for confounding control is still a black magic," #Bookofwhy

7.31.19 @2:25pm - (Replying to @EpiEllie and @BUSPHEpi) No one is really anti-DAG. But many propose extensions, enrichment and complementing methodologies. I was wondering if Dr. Krieger still believes the extensions she proposed in "Wagged by the DAG" are worth pursuing. (link: #Bookofwhy

7.31.19 @1:54pm - (Replying to @EpiEllie and @BUSPHEpi) Curious. Does Nancy Kreiger still upholds the views she expressed in: "The Tale Waggs the DAG?" (link: #Bookofwhy

7.31.19 @4:00am - Why I call it "upside-down culture"? Because the logical way to start is with what you KNOW, eg. structural equations, then use regression to estimate what you WISH TO KNOW. Confusion erupts when people think regression equations represent what you know. #Bookofwhy

7.31.19 @3:45am - (Replying to @TariqTaha123 @CNNSotu and @jaketapper) Straight from the book of slogans of BDS - the racist movement that shows no respect for truth or other people's identity. See (link:

7.31.19 @3:23am - (Replying to @willjharrison and @svenohl) Causation precedes manipulation. In #Bookofwhy we describe manipulation as a way of interrogating nature to reveal the causal forces that tie variables together.

7.31.19 @1:43am - (Replying to @isli_amar) Glad #Bookofwhy reached Algeria, probably from UK, swimming. I hope you start teaching it before young minds get frozen into "regression". The temporal constraints paper almost escaped my memory - so much has happened. Enjoy!

7.31.19 @1:30am - Link broken? Please try this one: (link:

7.31.19 @1:11am - In contrast, here is a new paper from political scienists (link: who address the upside-down culture of starting with "regression" and "controlling for" (eg Angrist's MHE) and then asking: Do our findings have any "causal interpretation?" #Bookofwhy

7.31.19 @12:53am - (Replying to @JamesLNuzzo and @NicoleBarbaro) Cofield's findings can be partially excused, for this was published 2010, only 10 years after epidemiologists started using graphs. But how come students in quantitative behavioral science do not rebel? #Bookofwhy

7.31.19 @12:17am - Very interesting paper. Published in 2019; 25 years after causal language has been mathematized and separated from statistical language! The author seems unaware of the causal revolution. No wonder! Has Psychometrika ever published an article on modern causal modeling?#Bookofwhy

7.30.19 @11:27pm - (Replying to @TariqTaha123 @CNNSotu and @jaketapper) My father was one of those Jews who came peacefully to his historical homeland. He was not silenced at all, I was there. He actually offered our Arab neighbors peaceful co-existence. Do you know what answer he got? I hate to embarrass you in public but, beware, I was there!

7.30.19 @9:44pm - (Replying to @JennieBrand1) Got it, thanks. Was I right in summarizing FE as: "An assumption of equality of two or more structural parameters which, in certain DAGs, leads to identification that otherwise will not be achieved." Q. The X's in your figs have no parents, what does it mean? #Bookofwhy

7.30.19 @9:04pm - (Replying to @JennieBrand1) Thanks Jennie, can you send an active link? Evidently, Oxford took UCLA off their university list, and they won't give me access to your paper.

7.30.19 @9:02pm - (Replying to @TariqTaha123 @CNNSotu and @jaketapper) I an going to show your Tweet when people ask me: What kind of neighbors Israel has? what kind of mentality drives them? and why it is so hard to reason with them? Dont they see that the two nations are equally indigenous to the land?My answer: see above "Me, Me, Me!"

7.30.19 @1:40am - (Replying to @maximananyev @Jabaluck and 3 others) Apropos:The more explicit the assumption, the more criticism it invites, for it triggers a richer space of alternative scenarios in which the assumption may fail. Researchers prefer therefore to declare threats in public and make assumptions in private. (link:

7.29.19 @10:41pm - (Replying to @Psylocke42356 @ZachWritesStuff and 2 others) This is precisely what I tell them in almost every piece I write. Can you reciprocate? Can you repeat after me (and ask Rashida to join): "Two states for two peoples, equally legitimate and equally indigenous". No ifs, no buts, just say "equally indigenous". Can you?

7.29.19 @10:10pm - (Replying to @Psylocke42356 @ZachWritesStuff and 2 others) Nations have a right to freedom and existence to the extent that they confirm such rights to their neighbors.

7.29.19 @9:58pm - One of the best quotes of the century. I hope it changes at least one heart.

7.29.19 @9:46pm - (Replying to @Jabaluck @fuzzydunlop123 and 2 others) That is why it is so important to write "The history of bad-control in pre-optics econometrics", to see precisely if the fumbling came from doubting the validity of models, or from inability to handle even a simple and valid model. Anyone writing? I'll help (anonymous) #Bookofwhy

7.29.19 @7:17pm - (Replying to @Jabaluck @fuzzydunlop123 and 2 others) And you insist you could have run a similar conversation with Angrist using the language of potential outcomes, where even "bad controls" are subject to embarrassment. BTW, is anyone writing the history of "bad controls" in pre-Telescopic econometric literature? #Bookofwhy

7.29.19 @3:13pm - (Replying to @fuzzydunlop123 @Jabaluck and 2 others) What am I watching? I thought DAGs are good for pedagogical purposes only. Now I see discussions on whether to condition on a variable or not...Dont tell me DAGs are good for discussions too, or for evaluating ID strategies, etc. etc. Beware of harsh consequences. #Bookofwhy

7.29.19 @2:31pm - Good news for causality research! Congratulations to Elias and co-authors for best paper award at the UAI-2019 conference, straight from Tel-Aviv marina, where I got my wind-surfing diploma in 1980 (framed in my office). They told me creative surfing is the secret to success.

7.29.19 @7:40am - (Replying to @ZachWritesStuff @CNNSotu and @jaketapper) What you teach your child is what you intend to do. Not one teacher, not two, but EVERY teacher. This is what you, as a journalist should decry and labor to change.

7.29.19 @7:31am - (Replying to @ZachWritesStuff @CNNSotu and @jaketapper) No one says Israel should not exist? You hav'nt heard what Palestinian teachers say repeatedly, nor what Omar Barghouti said at UCLA in 2014. See (link: .

7.29.19 @7:15am - (Replying to @glarange72 @CNNSotu and @jaketapper) The asymmetries on the ground are grim consequences of asymmetries in intention, with one side dreaming "We, We, We" and the other threatening "Me, Me, Me". Quite stark.

7.29.19 @6:19am - (Replying to @y2silence @stephensenn and 14 others) What is the simplest, canonical example of MLM showing that if you ignore the hierarchy you will not answer your research question properly. I am asking because the MLM papers I've chanced to read start and end with no explicit research questions. #Bookofwhy

7.29.19 @5:59am - (Replying to @ZachWritesStuff @CNNSotu and @jaketapper) States, indeed, just "exist", they have no rights. But when you tell your child: "Our neighbor has no right to exist" you are telling him: "We are not going to honor any peace agreement, forever". And you telling Israelis, you have compelling reasons to control those territories.

7.29.19 @4:50am - (Replying to @Vic1Nobody @ZachWritesStuff and 2 others) I am not a right-winger and I do not think it is fair to mention right-wing habits to shut down debates on core issues: Can Palestinians claim rights which they deny their neighbors. Current affairs are surface manifestations of this core issue.

7.29.19 @4:35am - (Replying to @shanbhardwaj @CNNSotu and @jaketapper) How? By tuning in, day and night, to what Palestinian leaders, clergy, and educators are saying (to their constituents, in Arabic) as I do, and my friends in the Israeli peace camp do, hoping to detect a seed of acceptance. Thus far - Nada.

7.29.19 @4:25am - (Replying to @glarange72 @CNNSotu and @jaketapper) Hypothetical questions have practical implication. In our case the implication is that one cannot demand rights to Palestinians that they deny their neighbors.

7.29.19 @2:51am - (Replying to @glarange72 @CNNSotu and @jaketapper) My position is that Jake Tapper should have asked her: "What should Israelis do if Palestinian leaders tell them what they do (I hope you heard them the past 75 years)???"

7.29.19 @2:40am - (Replying to @yudapearl @CNNSotu and @jaketapper) Poor Rashida, she just lost 90% of her voting base by saying: "Of course, but..." when Jake Tapper's asked her: "Does Israel have the right to exist?" It's tough to be a Zionophobe on CNN, torn between viewers norms of justice and voters push for elimination.

7.29.19 @1:40am - (Replying to @CNNSotu and @jaketapper) The punchline was missing: "What if the overwhelming majority of the Palestinians, leaders, educators and clergy, deny the right of Israel to exist, and openly declare that, occupation or not, they will to continue their arm struggle against Israel, in any borders?"

7.29.19 @1:32am - (Replying to @ZachWritesStuff @CNNSotu and @jaketapper) Zachary, Serious Person, associates a people's right to self-determination with "right-wing talking points". A serious person indeed.

7.29.19 @12:19am - (Replying to @AdanZBecerra1 @Lester_Domes and 14 others) I fail to see the relevance of MLM in this paper. I see constancy of effects across time, fine, but I do not see clusters or Set-Subset relationships in the examples. My blindness?

7.29.19 @12:03am - (Replying to @stephensenn @Lester_Domes and 13 others) I see no "strong undeclared assumption" in #Bookofwhy, perhaps because I read assumptions from DAGs and, looking at Fig.6.9, I see no assumption left out. Nada! The point is: {Data + DAG} determine which analyst was correct. No need to recruit Nelder to make this simple point.

7.28.19 @9:57pm - (Replying to @Lester_Domes @melb4886 and 13 others) Glad we are in agreement. I could never understand why certain folks, who are reluctant to learn causal inference, always excuse themselves with "we can do it in MLM". What is it in MLM that gives people the illusion that it can answer causal questions? #Bookofwhy

7.28.19 @9:09pm - (1/ ) Great paper. First time I understand what "fixed effect" is. I used to confuse it with homogeneity, but Fig. 1 tells us it is an assumption of equality of two structural parameters which, in certain DAGs, lead to identification that otherwise will not be achieved. So simple,
7.28.19 @9:09pm - (2/ ) "Why didn't they tell us?" I should complain, but I won't, because it was probably all there, in the papers and the lengthy motivations, and the indexed regression equations that I was too lazy to digest. Glad it is over, one figure did it. #Bookofwhy

7.28.19 @4:21pm - You are right. In case other readers of #Bookofwhy got stuck on Eq. 7.2 page 227, replacing Z with U would make it less mysterious in the context of the Z=Tar story. I hope we can make the change before the paperback edition hits the shelves. Thanks @the_aiju

7.28.19 @4:00pm - (1/2) I love your Advisor, and I think his "hide it" was an honest expression of the prevailing culture. I am sure that if he and colleagues start using DAGs in hiding, under the cover of "only for education," they will eventually use them everywhere, from seeing assumptions, to
7.28.19 @4:00pm - (2/2) to testing assumptions, to validating ID strategies, to discovering new ID strategies. Recall, IV was discovered using Wright's DAGs. Why? Because "using DAGs" simply means "using what you know". I presume the Church lifted its ban on telescopes "only for education"#Boodofwhy

7.28.19 @2:06pm - (Replying to @nyarlathotepesq) d-separation is valid for linear CYCLIC models as well. So it is easy to identify bad controls in simultaneous eq-ns. #Bookofwhy

7.28.19 @5:41am - FYI, I could not resist answering a new Quora question: "How does the Rubin causal model differ from graph-theoretic approaches like Pearl's do-calculus?" (link: There is nothing new there that our readers do not know; it is just compiled succintly #Bookofwhy

7.28.19 @5:34am - (Replying to @thosjleeper @PHuenermund and 4 others) No surprise that this is how IV is (still) taught in econ. They are talking about "structural equations" and call it "regression" and the variables "regressors". It is OK, as long as: (1) Econ. students can endure the confusion, and (2) a Glossary is in the making.#Bookofwhy

7.28.19 @3:51am - (1/ ) (Replying to @PHuenermund @dlmillimet and 3 others) In looking over other tweets in this thread I got the feeling that many of the questions are answered in the #causalinference literature, but the jargons are almost incompatible. I hope you plan on writing a glossary one day. For example, 2sls is an estimation method, hence it
7.28.19 @4:00am - (2/ ) (Replying to @yudapearl @PHuenermund and 4 others) has nothing to do with interpretation or with coefficients. Also, in the IV model the IV variable is the only "exogenous" variable. Finally, in a regression model "exogeneity" is not defined. The glossary seems like an endless job, but where would we be w/o it?#Bookofwhy

7.28.19 @2:20am - (Replying to @PHuenermund @dlmillimet and 3 others) @PHenermund I ventured to read your post and met your question: "How can we be sure that what we're estimating for the compliers is representative for the whole population?" Why don't Balke's bounds provide a general answer to your question? eg: (link: #Bookofwhy

7.28.19 @12:08am - (Replying to @stephensenn @AdanZBecerra1 and 12 others) Is there room for single-level reasoning anywhere in science? If not, I'll scrap 99% of my science books. If yes, I would first cast Lord's and Simpson's dilemmas in single-level context, see if I can solve them, then proceed to multilevel if needed, but only if needed.#Bookofwhy

7.27.19 @8:30pm - (Replying to @AdanZBecerra1 @stephensenn and 12 others) I am not questioning the need to do multilevel modelling when needed, be it in causal or predictive tasks. I am questioning the wisdom of forcing multilevel modeling on single-level causal questions (eg Lord's and Simpson's examples) w/o the tools of causal modeling. #Bookofwhy

7.27.19 @5:08pm - (Replying to @FJnyc @intelligence2 and 11 others) Mehdi Hasan will continue to stage these Kangaroo debates until someone charges him with "Zionophobic bigotry" in front of an audience, which would corner him to compare his perpetual denial of Jewish identity as people with his perpetual whining of "Islamophobic bigotry."

7.27.19 @3:49pm - (Replying to @AdanZBecerra1 @stephensenn and 12 others) @AdanZBecerra1 I do not think one should delve into multilevel models before acquiring the tools of handling single-level problems, like the one posed by Lord, or Simpson, or #Bookofwhy. Multilevel modeling, if imposed on every single-level problem can become counterproductive.

7.27.19 @3:11pm - (Replying to @matt_vowels) We are now hearing a 9th myth "A causal model is a special case of a predictive model". This one is particularly misleading, for it tells stat. students: "Dont bother to change your thinking, what we have been doing this past century is sufficient". No it ain't. #Bookofwhy

7.27.19 @3:02pm - (Replying to @marcelogelati) I can't decipher what they are trying to say, but if the're saying you can't infer causality from observationial studies alone, w/o a model, they are right. If they deny the power and ubiquity of qualitative models -- they are wrong. #Bookofwhy

7.27.19 @2:23pm - (Replying to @intelligence2 and @EWilf) Thanks for posting. I would never debate under such title, which pre-fixes its verdict. Show me one titled: "Is Anti-Zionism RACISM on its own merit" and I will show you a YES verdict earned. That is why I always use "Zionophobia", not anti-Semitism, e.g.

7.27.19 @4:37am - (Replying to @laurencepearl) It used to be Perl (on my grandfather visa, as he arrived at the Holy Land in 1924, from Poland). Legend says Jews bought their surnames centuries ago, from their European masters, based on their professions and the money they could raise. Seems Jewelry was a good profession.

7.26.19 @7:10pm - (Replying to @_asubbaswamy) Now I see it. But why did it take me 24 hours? Because I was missing one word: "E[T|do(a), c] is stable and IDENTIFIABLE". Naive me thought you are recommending an experiment with do(a). I suggest you stress this point explicitly in future papers.#Bookofwhy

7.26.19 @3:49pm - (Replying to @laurencepearl) Don't understand!! ??? You must be kidding! How do you estimate causal effects? Or, how do you find stable predictors? More importantly, how do your colleagues do that? You'r kidding!!! #Bookofwhy

7.26.19 @3:41pm - (Replying to @_asubbaswamy) Thanks for the refinement! Agree. S goes into X. A question to you: Why didn't you use a simpler example than Fig. 2(b) to show advantage over "pruned estimators" ?? It took me an hour to believe that such examples exist. #Bookofwhy

7.26.19 @3:06pm - Just occurred to me: Is't the front-door a simple example of getting an unbiased and stable estimator of Y, given observations on X and Z (with U varying)? [ X--->Z--->Y with confounder U, X<-U->Y. ] E[Y|x,z] is unstable, whereas the front-door formula is. #Bookofwhy

7.26.19 @2:43pm - (1/ ) Saying that "a causal model is a special case of a predictive model" is like saying "sailing is a special case of swimming, since it is conditional of something floating". In general, saying that task-A is a special case of task-B depends on what the sayer is trying to get:
7.26.19 @2:43pm - (2/2) an excuse for not doing A, or a license for doing A using methods used in B. The latter would be justified, if it was possible. Unfortunately, causal models require information (+methods) not available in traditional prediction modeling. So why say "special case"? #Bookofwhy

7.26.19 @2:19pm - You are right, the word "Zionophobia" is gaining traction, including the Justice Department Summit, see (link: It is becoming "the ugliest word in town", at least among people of conscience. It's the most effective defense weapon I know.

7.26.19 @6:00am - (Replying to @suchisaria @KordingLab and 4 others) The idea of "surgery estimators" is ingenious, it would not occur to me that you can get extra mileage on top of "pruned estimators". However, is Figure 2(b) the simplest example to show this extra mileage?? Would c-equivalence help here? (link: #Bookofwhy

7.26.19 @5:44am - (Replying to @nyarlathotepesq) You are right, the bet is not a proof, that is why I suggested an Appendix with some proofs. But the two polar poles is not a good analogy; going from linear to nonparametric amounts to monotonically removing constraints. Are you considering writing it? #Bookofwhy

7.26.19 @3:10am - (1/2) Great question! Three answers. 1) In practice, we are really seeking "safe control", not "bad control" and nonparametric analysis gives it to us. 2) If something is bad in both nonparametric and linear analyses, you can bet it is bad in between. 3)
7.26.19 @3:10am - (2/2) (3) All the fumbling and stumbling I have seen in the econometric literature occur already in linear systems, where the proof of "badness" is easy, see (link: Plus, you can discuss the question in the Appendix. #Bookofwhy

7.26.19 @2:07am - (1/2) BAD-CONTROL. If I were a young economist, seeking visibility and impact in my field, I would sit down and write an article titled: "Bad-Control - A lingering challenge and its resolution." I got this thought upon reading (link: noting that Angrist (2017)
7.26.19 @2:07am - (2/2) still calls this elementary econ. exercise: a "difficult problem". Such article will be a highly appreciated eye-opener to many of your peers, but it must be written diplomatically -- lot's of professional honor involved. #Bookofwhy

7.26.19 @12:34am - (Replying to @neurosutras and @JonAMichaels) Causal thinking does not mean purging predictive and associative relations; it means using such relationships properly whenever they emerge from a causal model of the world. #Bookofwhy

7.25.19 @6:38am - (Replying to @KordingLab @fhuszar and 4 others) Everything Econs did is correct, but what they did is of limited scope. I once asked: how many economists can do X, or Y etc. and I lost all my econ friends. See Causality p.216 fn10 - I will not repeat this mistake - do not force me, please. #Bookofwhy

7.25.19 @6:23am - (Replying to @brunofmr and @suchisaria) "Stability" under environmental change should not be confused with "stable distribution" which is a purely probabilistic notion, and has nothing to do with environmental changes. I hope no confusion results. #Bookofwhy

7.25.19 @5:03am - (Replying to @suchisaria @KordingLab and 4 others) Will do.

7.25.19 @5:02am - Not many people realize that the strength of a DAG comes from building ALL its logic on ONE primitive question: "Why does a variable vary?" All the rest is mechanically derived, demanding no further judgment. Poetically, I crowned it: "going where knowledge resides." #Bookofwhy

7.25.19 @4:44am - (Replying to @KordingLab @fhuszar and 4 others) I dont see it this way, since I hardly touched on "estimation" (ie, going from finite sample to distribution); I call the stat dept once I get an estimand. Econometrics has been doing some limited identification correctly. How limited? See: (link:, #Bookofwhay

7.25.19 @1:26am - (1/ ) I just read (link: and I agree with @suchisaria . Anyone concerned with stability (or invariance) should start with this paper to get a definition of we are looking for and why. Arjovsky etal paper should be read with this perspective in mind, as an attempt to
7.25.19 @1:26am - (2/ ) secure this sort of stability w/o having a model, but having a collection of varying datasets instead. The former informs us when the latter's attempts will succeed or fail. It is appropriate here to repeat my old slogan: "It is only by taking models seriously that we learn
7.25.19 @1:26am - (3/ ) when they are not needed". I wish I could quote from Aristotle but, somehow, the Greeks did not argue with their Babylonian rivals, the curve-fitters. They just went ahead and measured the radius of the earth AS IF their model was correct, and the earth was round. #Bookofwhy

7.24.19 @4:03pm - (Replying to @fhuszar @yudapearl and 3 others) It makes sense to me to start from what is it that we know must hold true. Discovering which invariances you want to guarantee based on your data is OK but the data are only a sample and it's important to declare which invariances are desired and why.

7.25.19 @12:44am - (Replying to @RichmanRonald) This paper (link: .is an interesting window into statistics education of 2019. Contrary to causal logic, statistics students start with data visualization routines and then ask: when do these have causal interpretation. I hope #Bookofwhy will change this order.

7.25.19 @12:19am - (Replying to @KordingLab @fhuszar and 4 othersz) There has always been an understand between ci and ml -- I do identification and you do estimation. Nothing has changed with all the talk about "causal ML", except perhaps ML folks internalizing the limits of the Ladder of Causation #Bookofwhy

7.24.19 @6:20pm - (Replying to @Jabaluck @PHuenermund and @fuzzydunlop123) I happened to pass by this thread. There is no testable implication for IV from observational studies unless treatment is discrete. See (link: #Bookofwhy

7.24.19 @6:03pm - (Replying to @suchisaria @tdietterich and 2 others) Yes, these papers are in scientific language. Not because they are using DAGs, but because they provide theoretical guarantees under meaningful assumptions. I've vowed to read them and comments. #Bookofwhy

7.24.19 @4:22pm - (Replying to @RichmanRonald) Thanks for the pointer. I will try reading it tonight. It is always a valuable learning experience to see how statisticians think about causal problems.

7.24.19 @3:40pm - (Replying to @fhuszar @tdietterich and 3 others) How about adding it as appendix to your blog? "From ppp to SSS - a declarative summary"

7.24.19 @2:35pm - (Replying to @yudapearl @fhuszar and 4 others) Most importantly, having read Arjovsky etal paper, do we understand what ppp and SSS are? Or, at least what the claim is? Such translation will help evaluate the claim under the light of existing theories. #Bookofwhy

7.24.19 @2:25pm - (Replying to @yudapearl @fhuszar and 4 others) "If a pattern ppp is seen in the data then something (SSS) must hold true in the world". The language of "my algorithm tries" of "my algorithm optimizes" etc reminds me of AI in the 1970's "MYCIN tries" "ELIZA optimizes" which was later replaced with declarative writing style.

7.24.19 @2:09pm - (Replying to @fhuszar @tdietterich and 3 others) Agree, but at this point we are trying, not to convince ML folks but to understand what their new method provides, and under what assumptions. To understand what a ML paper offers we need someone to translate the paper into a declarative language, that goes:

7.24.19 @5:23am - For those interested in the history of UAI, I wrote a personal memorandum of those days (link: #Bookofwhy

7.24.19 @4:56am - (Replying to @yudapearl @fhuszar and 4 others) we would say: "If we observe x,y pairs from multiple E's, and we find Y||E|X,W and NOT-X||E|Y,W, then something must hold true in the world." Now, let's continue from here: What is it that must hold true? Can you get it from the paper? #Bookofwhy

7.24.19 @4:50am - (Replying to @fhuszar @tdietterich and 3 others) Your summary is ALMOST Causal language, but not quite. In Causal language we do not invoke man-made algorithms (IRM) to describe environmental properties such as invariants; we speak declaratively. For example, instead of saying: ""IRM observes" x,y pairs from multiple E's,"

7.24.19 @4:34am - (Replying to @sobu_18) No, No, the word Zionophobia has a unique magic to it, unmatched by antisemitism. I've never met an anti-Semite who admitted to being one, and I've never met a Zionophobe who denied being one. Quite a difference for a mentality that denies a homeland to a people.

7.23.19 @11:38pm - (Replying to @suchisaria @tdietterich and 2 others) Great! I was't aware of these two papers. Now we know that searches for "stability" and "invariants" are on firm scientific grounds: When we suspect a certain relation is "stable" we can check and see that it is truly stable and what conditions will make it unstable.#Bookofwhy

7.23.19 @8:59pm - Sorry to have missed it, hoping a video was taken. I always regret not having a video from the first UAI, 34 years ago, at UCLA-1985, when probabilities first infiltrated the forests of AI. I see causality following a similar route. #Bookofwhy @eliasbareinboim

7.23.19 @7:02pm - (Replying to @tdietterich @zittrain and 2 others) I understand that IRM leverages the availability of multiple data sets, usually absent from CI. So let's represent the multiple data sets in the language of CI and see what must hold in the world for IRM to find the invariants it is searching for. Anyone done it? #Bookofwhy

7.23.19 @3:54pm - (Replying to @sobu_18) Careful reading of (link: reveals that #BDS is not charged with antisemitism, but with Zionophobia: The irrational animosity toward a homeland for the Jewish people. Populist slogans like "apartheid" tend to tarnish the credibility of their chanters.

7.23.19 @3:08pm - (Replying to @tdietterich @zittrain and 2 others) @tdietterich, Since you are familiar with both CI and Invariant Risk Mininization (IRM), can you (or anyone else) explain in input-output terms how IRM extracts from non-experimental data information that CI thought must be obtained from either a model or intervention. #Bookofwhy

7.23.19 @3:59am - (Replying to @pcastr) If you take seriously my experience with BDS activists (link: you will see that "boycott" is the last thing on their agenda. Their aim is to silence pro-coexistence voices. Is it really tenuous to see that absent such voices Israel's demise is almost certain?

7.23.19 @3:12am - I, likewise, just want to voice my conviction that support of BDS may have serious unintended consequences, a genocidal demise of Israel is one, as I describe here: (link: From there to the demise of the Jewish people takes another leap of logical deduction.

7.23.19 @12:34am - If you know a BDS promoter who bought into its rhetoric without checking the destructive aims of its leadership, you have found someone who has not studied BDS as thoroughly as I have. See (link:

7.22.19 @8:25am - (Replying to @azuur @acastroaraujo @Jabaluck) I beg to differ. It discards whatever does not have a MODEL, which need not be a DAG. Since every CI study must rely on a model, not having one means keeping one in your head and giving the impression that you operate model-free. Both are ill-advised. #Bookofwhy

7.22.19 @8:16am - (Replying to @azuur @acastroaraujo @Jabaluck) Completamente de acuerdo!!

7.22.19 @7:47am - (Replying to @bzaharatos @learnfromerror) I am also going to miss JSM this year (Carlos Cinelli will represent me at the Fellow Reception). As to Philosophy+Statistics seminar, glad someone will be there who can translate back to us, down in the trenches, speaking cause and effect. #Bookofwhy

7.22.19 @6:16am - (1/ ) (Replying to @RonKenett @learnfromerror) 1/ I am not sure that we are talking about the same notions of "generalizability". For me, this word means taking experimental results from one population and applying it to another, potentially different. I am not sure @learnfromerror means the same thing; I would be
7.22.19 @6:22am - (2/ ) (Replying to @yudapearl @RonKenett @learnfromerror) curious to know if she does. Why I am not sure? Because the way the two populations may differ may be non-statistical, namely they may have the same joint distribution functions on all variables, and differ ONLY in the causal forces holding the variables together #Bookofwhy
7.22.19 @6:30am - (3/ ) (Replying to @yudapearl @RonKenett @learnfromerror) Under such circumstances statistical methods cannot deliver remedy unless they are guided by a causal model of those underlying forces. Such models are absent from standard statistical writings, they are introduced back-door in the PO literature, as in

7.22.19 @1:20am - (1/ ) (Replying to @PHuenermund) Thanks Paul for starting this important discussion on @EconBookClub of Imbens paper. Since the paper is full of objectionable macro and mini-statements, I believe it is wise to focus on its core, summarizes precisely in the paragraph you posted. "Little is said about what 7.22.19 @1:30am - (2/ ) (Replying to @yudapearl @PHuenermund) comes before the identification question and what comes after the identification question" . The first reflects misunderstanding of what structural economics is about, while the second decries a "before vs. after" distinction that should be welcome with joy. The rest of Imben 7.22.19 @1:39am - (3/ ) (Replying to @yudapearl @PHuenermund) paper follows as a corollary of these two basic misunderstandings. Once we illuminate these two, it should be easy to clear the rest, especially the identification stage itself, which he admits to be virtually absent from his tool set. (if # of variables exceeds 3) #Bookofwhy

7.21.19 @11:47pm - An old saying goes: "When mathematicians notice an interesting problem it becomes SCIENCE". Today, Notices of The American Mathematical Society took notice of #Bookofwhy, thanks to Lisa Goldberg : ... I hope #causalinference gets enriched with new insights.

7.21.19 @11:18pm - This is funny! Cornel West is "standing in moral solidarity with four sisters?" Last I wrote about him and his moral deformity, it was he who needed moral solidarity ...

7.21.19 @10:45pm - (Replying to @bzaharatos) In what way?

7.21.19 @9:34pm - (Replying to @oacarah) On the mark! Which proves the algebra is for the birds.

7.21.19 @6:02pm - Your daughter, Isadora, is adorable. If she could only get that partial differential equation right, we could have had a perfect afternoon -- next time!

7.21.19 @4:49pm - Look Ma! I'm a statistician! They tell me that at the upcoming JSM meeting in Denver I'll be ordained as an ASA Fellow. I assume it means that, starting July 30, statisticians can treat #Bookofwhy as a homegrown tomatoe, and each of its toy problem as 10,000 "real-life" examples.

7.20.19 @3:01pm - (Replying to @AdanZBecerra1 @PHuenermund) Do you know what blog or forum do thoughtful statisticians use to communicate ideas about the philosophy of statistics, including causal inference ??

7.20.19 @4:41am - (Replying to @AdanZBecerra1 @PHuenermund) Another remarkable observation: none of the discussants had any clue on how to handle conditioning on post-treatment variables. The advice they got was: "Control for as many pre-treatment variables as you can." Is this the best place to learn how statisticians think? #Bookofwhy

7.19.19 @11:37pm - (Replying to @yudapearl @AdanZBecerra1 @PHuenermund) @AdanZBecerra1, I remember you from Gelman's blog. You reminded the discussants of DAGs and got bullied by one of the big guys who told you to go solve a "real life" problem. I was tempted to come to your rescue but those guys knew so much about "real-life"- got scared #Bookofwhy

7.19.19 @8:11pm - (Replying to @PHuenermund) Totally agree. Discussions should start with the question: Why are DAGs "pedagogically" more "transparent". Is it just "taste"? Or is it something universal in our minds that makes DAGs good "displayers of assumptions"? The rest follows from this cognitive phenomenon.#Bookofwhy

7.19.19 @7:37pm - (Replying to @PHuenermund) Interesting. Guido may have changed his mind - I did not. I truly believe that if any of my economist colleagues actually roles sleeves and solves a couple of toy problems, instead of talking ABOUT them, he/she will never go back to talking ABOUT them. #Bookofwhy

7.19.19 @12:28pm - Each toy problem is 10,000 "real life" examples, in which you cannot hide behind messy data, unobserved confounders, or other excuses and, moreover, you can check your method against ground truth. Those who shun toy problems do have something to hide, watch them. #Bookofwhy

7.19.19 @2:23am - (Replying to @aqsaqal) Sounds like a beautiful example of missing-data. And I bet it is recoverable, since the missingness of Weight is not caused by Weight itself. See a graphical approach (and stay away from imputation) #Bookofwhy @Carthica

7.19.19 @1:39am - (Replying to @thosjleeper @matloff) My My! Good point! It is better that they leash out on me than to let truth reduced to "gossip". Thanks, #Bookofwhy

7.18.19 @11:32pm - (Replying to @yudapearl @thosjleeper @matloff) Why is quoting unnamed statisticians a bad thing? Would naming them make less hostile? Recall, they are still alive, deciding on promotions, and perhaps repenting for what they once thought. Why embarrass them? #Bookofwhy

7.18.19 @11:27pm - (Replying to @thosjleeper @matloff) Comments on your thread. The statement P(Y|do(x)) is causal but not counterfactual, because it implies no contradiction, and it can be estimated from RCT. P(Y|see(x)), on the other hand, is statistical, not causal, because we can evaluate it without experiments. #Bookofwhy

7.18.19 @11:12pm - (Replying to @anirudhacharya1 @oacarah) How is this: ?? Any luck? #Bookofwhy

7.18.19 @10:56pm - (Replying to @klausmiller @PHuenermund and 3 others) Judea Pearl Retweeted Judea Pearl As I Tweeted here: ... this is the greatest thing that happened to DAGs: Sunrays are the best de-confounders. #Bookofwhy @quantadan @analisereal

7.18.19 @10:39pm - I like this title: "When in Doubt, DAG it Out. " @oacarah new commentary on "Analyzing Selection Bias for Credible Causal Inference" ... It confirms my mantra: "DAG goes where knowledge resides," which PO folks are about to internalize. #Bookofwhy

7.18.19 @8:11pm - (Replying to @vauhinivara @WSJ) Delighted to read your thread and to see that the WSJ Internship set up in memory of my son Daniel was instrumental in lifting you forward. Pitching story, so it seems, is much like pitching scientific articles - enlightened editors are scarce. Good luck #Bookofwhy @craigmatsuda

7.18.19 @2:41pm - Thanks for pointing to Imben's new paper on PO and CI. We finally have a window into the thinking of leading PO researchers. It will give econ. students a chance to compare alternatives and ask: Do we really want to think that way? #Bookofwhy @StatsPapers @causalinf @PHuenermund

7.18.19 @12:16am - (1/2) (Replying to @omaclaren @djvanness and 2 others) 1/2 We have this world; it is called classical mathematical modeling. What this world was missing (when I last fiddled in it) was the EXTREME case called DAGs. Namely, the miracle of how much can be accomplished with so few and weak assumptions. I might go back to classics, once
7.18.19 @12:22am - (2/2) (Replying to @yudapearl @omaclaren and 3 others) we fully understand the full potentials of this new miracle. But it seems to go on and surprise us with new capabilities: external validity, missing data, sensitivity analysis, fairness,...Its pouring! And you want us to quit? #Bookofwhy

7.17.19 @10:53pm - (Replying to @omaclaren @djvanness and 2 others) You have this strength in SCM if you want to express something you are really sure about. But if you are only sure about "who listens to whom" use DAGs. Perhaps you are after a calculus that works in between these two extremes? Happy sailing; watch the two extremes #Bookofwhy

7.17.19 @10:43pm - (Replying to @omaclaren @djvanness and 2 others) Easy! You write Y = OR(X,Z) or Y=AND(X,Z). This is how you express it if you think you can. If you can't commit to one function or another you just write Y=f(X,Z) and you let data decide. If your data must be taken in isolation, it is mathematical impossibility.#Bookofwhy

7.17.19 @10:35pm - (Replying to @omaclaren @djvanness and 2 others) You are repeating what I said, so we are settled. Recall the DAGS were devised to minimize the amount of information (ie assumptions) demanded from researchers, and limit it to those relations only that reside in the scientist's comfort zone --variable listening #Bookofwhy

7.17.19 @10:20pm - (Replying to @LMCheongTS) RL can handle some interventional problems, like predicting effects of actions tried before. As to counterfactuals, RL can potentially produce bounds on counterfactuals, the same as what Rungs 1 & 2 produce. A miracle shown here: . #Bookofwhy

7.17.19 @10:11pm - (Replying to @omaclaren @djvanness and 2 others) Yes, `interaction' a concept that is formalised in SCMs . Compute P(Y|do(x),z)) and check if it depends on z. Note that this quantity is computable from full SCM specification. Data is needed only to supplement missing specifications, say P(u), or functional forms #Bookofwhy

7.17.19 @10:04pm - (Replying to @omaclaren @djvanness and 2 others) To state directly that interaction does not exist SCM must resort to some parametric specification, say "linear family". Or, indirectly, SCM can tell us: "Check for yourself. I have given you part of me, a DAG, which implies a recipe for interrogating the data; use it. #Bookofwhy

7.17.19 @9:56pm - (Replying to @ntabari8221 @fulhack) The #Bookofwhy argues that too much time is spent on "confounders," What we should be looking for are the "de-conounders".

7.17.19 @9:52pm - (Replying to @djvanness @ConiByera and 2 others) Yes, it is "legal" with or without the U. And, remarkably, one cannot tell who is the modifier and who is the "cause". This distinction still demands a "penalty" definition: What would I lose if I incorrectly switch the two. #Bookofwhy

7.17.19 @9:45pm - (Replying to @djvanness @ConiByera and 2 others) Not really. The topology of the DAG (with or w/o U) tells us how to get the Z-specific causal effects. Whether or not those effects depend on Z would be revealed from the data, once we estimate P(Y|do(x), z) correctly. #Bookofwhy

7.17.19 @8:33pm - (Replying to @djvanness @ConiByera and 2 others) The last warning is given to you by the DAG, coupled with a method of quantifying the penalty. So, what is missing? #Bookofwhy

7.17.19 @8:30pm - (Replying to @djvanness @ConiByera and 2 others) I am still missing what that "additional extra-statistical structural information" is that you need "to get the policy estimates right." What kind of answer you expect the system to deliver to you if you had this extra information encoded in the DAG #Bookofwhy

7.17.19 @8:25pm - (Replying to @djvanness @ConiByera and 2 others) If your target quantity is P(Y|do(x),z) for different z's, you can identify it by controlling for C. The DAG with U gives you the same information, ie "interaction is possible". You probably want an early warning saying: "dont bother, no interaction here". Right? #Bookofwhy

7.17.19 @8:16pm - (Replying to @djvanness @ConiByera and 2 others) If all you wish is P(Y|do(X=1)) then you do not need to account for Z. But you really wish to know more. You must start by telling the inference-engine what it is. Perhaps you seek the difference P(Y|do(X=1), Z=0) - P(Y|do(X=1), Z=1)? Or ratio? or something else? #Bookofwhy

7.17.19 @8:06pm - (Replying to @djvanness @ConiByera and 2 others) I will try to answer slowly each one of your tweets. With DAGs, no one is left confused. The DAG you drew just says: I don't rule out interactions between Z and X. And he ( Mr. DAG) is now waiting for you to specify you research question. What is it? #Bookofwhy

7.17.19 @7:51pm - (Replying to @omaclaren @djvanness and 2 others) SCM assumes the existence of functions Y=f(x,z,w...), with the help of which we define other concepts such as causal effects, explanation etc. Whether or not we can identify those concepts from data is a separate issue. Those who wonder whether DAG has a concept C need first to
7.17.19 @7:59pm - (2 )(Replying to @yudapearl @omaclaren and 3 others) define what they mean by C. They can use mathematics, or examples, or the method of "penalty", which goes: "If I know C, I can escape from the penalty that I would suffer not knowing C." So far I have not seen a penalty definition of the concept sought by Dave. #Bookofwhy

7.17.19 @7:43pm - (Replying to @mgaldino) I care less about those "some economists" than I care about our "current and living economists". Do they tell their students: "We rely on models"? Are they proud or ashamed of using models? Are they reading models or hiding them? @Bookofwhy

7.17.19 @6:39pm - Every time I read Quora I wonder: Do empirical economists in Angrist etal school consider themselves "model-based" or "model-free". Some say: "unlike statisticians, we use models". Others say: "unlike traditional econs, we are not tied to models". I wish someone would clarify.

7.17.19 @6:17pm - (Replying to @djvanness @ConiByera and 2 others) I tried to follow this thread, and still can't see what "logical interaction" you wish to represent. DAG in itself just says "there can be an interaction". SCM says more: it tells us the function Y=f(X,Z,U,W...). But I sense that you want more, what is it? and what for?#Bookofwhy

7.17.19 @5:12am - Apropos "Zionophobic Thuggery", I am delighted to learn that the words are gaining traction all the way to the Department of Justice. They were mentioned 5-6 times in this Summit meeting of DOJ , and they are making my students vow: No more of this racism.

7.17.19 @4:50am - And I need to check my blog, to make sure I ever wrote it. Who knows how many useful nuggets are buried in that blog, written before Twitter, when I had time to write long answers, and when people expected in-depth answers to questions that textbooks tended to ignore. #Bookofwhy

7.17.19 @3:41am - Funny, 1/4 of a century later, the introduction to "Graphical Models, Causality, and Intervention" (1993) sounds like it was taken straight from #Bookofwhy

7.17.19 @2:33am - The first paper using do() notation was "A probabilistic calculus of actions" (1994) , though the idea was used earlier (1993), using set(X=x) notation, in #Bookofwhy

7.16.19 @12:39am - (Replying to @djvanness @y2silence and 2 others) Good example where most people agree that Z is a modifier of the effect of X on Y, and not that X is a modifier of the effect of Z on Y. So, back to penalty analysis. What will go wrong if I mislabel them and I mistake X to be a modifier of the effect of Z on Y? #Bookofwhy

7.16.19 @12:26am - (Replying to @VladZamfir @lostinio and 3 others) Curious, what is your favorite epistemology of uncertainty? Plus, what is a good way to assess the merit of such epistemology? #Bookofwhy

7.15.19 @9:56pm - (Replying to @VladZamfir @oliverbeige and 2 others) Two remarks. (1) There are hordes of identities one can extract out of probability theory. Why did Bayes' receive such prominence? (2) I am still searching for a definition of Bayesianism, not for what's wrong with its practitioners. #Bookofwhy

7.15.19 @8:41pm - (Replying to @djvanness) The way I usually get to the "essence" of a distinction is to ask myself: How would I be penalized if I did not know the distinction. In our case: What would I do wrong if I were to mistake an "effect modifier" for a "cause"?? From the penalty comes the "essence." #Bookofwhy

7.15.19 @6:20pm - (Replying to @djvanness) Many feel that DAGs and other models do not capture the "essence" of "effect modifier", as opposed to just "cause". But few have taken time to explicate, even semi-formally what that "essence" is, and how they would benefit if a model does capture it. Care to try? #Bookofwhy

7.15.19 @3:28pm - (Replying to @VladZamfir @oliverbeige and 2 others) The frequentist interpretation of Bayes rule is a one-line high-school algebra trick. This is not what made Bayes famous, controversial and revered. #Bookofwhy (p. 102-3) proposes an explanation of what made Bayes rule justifiably revered.

7.15.19 @2:48pm - (Replying to @ipyadev @_MiguelHernan @JohannesTextor) Thanks for posting this paper, by Clarice, which skipped my awareness. The key sentence is: "Any 2 direct causes of D are effect modifiers for each other on at least 2 scales, which can make a reasonable person question the utility of the concept. " I'll try to clarify #Bookofwhy

7.15.19 @5:46am - (1/ ) (Replying to @oliverbeige @VladZamfir and 2 others) For some people, Bayesianism is spraying priors on parameters and waiting for the posteriors to peak. For others, especially those who read Bayes (1763), Bayesianism means 1) leveraging subjective knowledge and 2) processing evidence by Bayes' Rule. I was in the second camp
7.15.19 @5:55am - (2/ ) (Replying to @yudapearl @oliverbeige and 3 others) When I coined the name "Bayesian networks" (1985), but I am camp-friendly and would not mind switching if you can tell me what I am switching to. In the meantime I also wrote "Why I am only half Bayesian" which will affect my next camp #Bookofwhy

7.15.19 @12:42am - (Replying to @brettjgall @BrendanNyhan @NateSilver538) This is an under-understatement. Causal inference is actually a mathematical impossibility without a model, more accurately, without a "causal model", namely one that cannot be articulated in the language of statistics. #Bookofwhy

7.14.19 @9:51pm - (Replying to @ZionessMovement @CoryBooker) Thank you @ZionessMovement for joining my Twitter and for informing me of how @CoryBooker and VC Biden responded to IfNotNow squeeks. I neve say the Israeli Palestinian conflict is "COMPLEX". It is baby simple: One side says "we, we, we" and the other "me, me, me". Baby simple.

7.14.19 @7:54pm - (Replying to @ipyadev @_MiguelHernan @JohannesTextor) In DAGs all variables are presumed to be "effect modifiers" by the nonparametric nature of the assumptions. You probably meant to ask: How do we mark variables that are NOT effect modifiers. My question: for what purpose? ie What would those marks enable you to do? #Bookofwhy

7.14.19 @1:55am - (Replying to @ERMANigeria) In Quora we are witnessing another phenomenon, not parametric addiction. Angrist for example, ... talks as if economics has a unique & exclusive ownership on all CI questions, and everyone else is doing data-fitting. It's a remarkable phenomenon. #Bookofwhy

7.14.19 @12:19am - (Replying to @ERMANigeria) Please share. I have not seen that. Perhaps because, immersed in non-parametric modeling, I failed to notice what my colleagues have been doing. #Bookofwhy

7.14.19 @12:17am - (Replying to @henning_lars) I agree. But how do they intend to "aggregate data across a variety of sources," about which "good old statistical methodology" is totally helpless. [And I include Meta Analysis in this state of helplessness]. I am afraid they will fall irreversibly for same dead-ends #Bookofwhy

7.14.19 @12:03am - Sharing. I have noticed that all CI-related questions on quora are answered by econometricians who, as we have noticed on this Twitter, are not too familiar with SCM (yet). I therefore posted a new answer: , in the hope of stimulating awareness #Bookofwhy

7.13.19 @6:43pm - (Replying to @akelleh) Beautiful introduction to Inverse Probability Weighing of do-adjustment. I hope you have more users than argumentative skeptics. #Bookofwhy

7.13.19 @6:25pm - (Replying to @stuartbuck1 @asacarny and 3 others) I do not see any condescending comments around. I see a request to accept the premise that "graphs go where the knowledge resides". If accepted, the rest is corollaries. If not, lets discuss what relationships the mind may find easier to discern than "Y listens to X" #Bookofwhy

7.13.19 @5:30pm - Thanks for posting these quotes from Karlin and Pearson. Interestingly, today we hear a slightly different song: "Yes, the process generating the data is important," - end of song. No! It's the beginning: How do we represent that "process"? How do we operationalize it? #Bookofwhy

7.13.19 @4:39pm - (1/ ) Interesting News: Google sister-company Verily is teaming with big pharma on clinical trials ... Quotes: 1) Will try to find ways to modernize their clinical trials and speed up the time it takes to bring a new drug to market. 2) aggregate data across a
7.13.19 @4:39pm - (2/ ) variety of sources, 3) Clinical trials have historically been expensive processes that rely on outdated technologies. Does anyone know the technical leaders of this initiative? e.g., Are the versed in the less outdated literature on "aggregating data"? #Bookofwhy

7.13.19 @3:05pm - (Replying to @dkipb) It runs deeper, agree. It's a product of screwed up education in all fields. Today's urgency is to get over those tribal narratives and get things right. And to get things right we need accept that business is not as usual -- those distinctions must be operationalized #Bookofwhy

7.13.19 @5:56am - (Replying to @jack_bowdenjack @RobinGomila) Sure, but this does not make the difference a "causal effect". What makes it a "causal effect" is the model and what the model says about the ignorability condition, which is here taken to hold apriori, thus assuming away the causal component of the problem. #Bookofwhy

7.13.19 @4:13am - (Replying to @NoahHaber @economeager and 2 others) @NoahHaber, I overlooked your thoughtful thread on the comparison of epi to econ. Allow me to reiterate another element: Epi use DAGs because "DAGs go to where knowledge resides". Econ. do not use DAGs because it is culturally prudent to do things in your head. #Bookofwhy

7.13.19 @2:54am - I've just posted an answer on quora ... which ends with a catchy phrase of what #Bookofwhy is trying to do: It tries to "dispel the myth that AI is about to be colonized by statistics. " -- Sharing

7.13.19 @2:35am - I still can't figure out what makes statistics leadership so eager to tell their constituency: Don't panic, there is no confusion, our ancestors have been here before, they touched, they recognized and pointed out, they discussed and distinguished, business is as usual #Bookofwhy

7.13.19 @2:09am - (1/ ) (Replying to @RobinGomila) I think the sentence: "I draw on econometric theory and established statistical findings to demonstrate that linear regression (OLS) is generally the best strategy to estimate causal effects on binary outcomes" will make some economists rethink if "Almost Harmless" is truly
7.13.19 @2:20am - (2/ ) (Replying to @yudapearl @RobinGomila) as harmless as some claim. It seems to me that you are dealing with strategies to estimate conditional expectations, not causal effects. This potential confusion is prevalent in the new culture schooled by structure-less econometrics.

7.12.19 @2:57pm - This "Mediation Formula" always restores my confidence in the power of very simple mathematics to produce very meaningful results. #Bookofwhy

7.12.19 @2:38pm - (1/ ) (Replying to @asacarny @paulgp and 2 others) I am not an NBER member, but ocassionally producing results that are of interest to economists. The exclusive policy of NBER prevents its members from learning these results. This in itself may not be sufficient reason to change policies, but the reputation of econometrics
7.12.19 @2:44pm - (2/ ) (Replying to @yudapearl @asacarny and 3 others) as an insular and outdated field does hurt its members, and increasingly so in the age of data-science. I therefore welcome your proposal to open another channel of communication between economics and the rest of the scientific community. #Bookofwhy

7.12.19 @3:31am - (Replying to @yudapearl @AndroSabashvili @_MiguelHernan) Gee, I think I know why they discarded explanations; it goes back to the RCT's roots and the philosophy of "well defined interventions". You cannot define "undoing past events" unless you have manipulations; plain "events" cannot explain things, only manipulations can. #Bookofwhy

7.12.19 @3:21am - (Replying to @AndroSabashvili) I am not sure "explanation" is part of @_MiguelHernan taxonomy, because explanations demand retrospective counterfactuals, not "predictive counterfactuals". Thus, scientists quest for explanation remains outside data science. I wish I knew why they changed the ladder. #Bookofwhy

7.12.19 @3:02am - (Replying to @AndroSabashvili) You are very right. I wrote to @miguelHernan: 5.7.19 @1:47pm - I am questioning the benefit of separating "description" from "prediction", skipping "diagnosis" and lumping together "intervening" and "retrospecting" under one opaque category "causal inference". No-Ans. #Bookofwhy

7.11.19 @10:21pm - (1/ ) (Replying to @PavlosMsaouel) 1/ Glad I found another fan. Shmueli's paper had many (1,200) citations which is encouraging. Statisticians hit the roof when I tell readers how causality was neglected in 20-century statistics. Glad they have accepted Shmuei's story. It seems that David Hand wrote his new paper
7.11.19 @10:40pm - (2/ ) (Replying to @yudapearl @PavlosMsaouel) to refute Shmueli's conclusions but, in my opinion, he actually reinforces them with additional evidence provided by the refs to Box, Breiman (2001) and Cox (1990). I hope #Bookofwhy sparks renewed discussion on the nature of modeling.

7.11.19 @8:43pm - (1/3) While struggling to find a ladder or demarcation lines between rungs, Hand's paper called my attention to a comprehensive article by Shmueli (2010) "To Explain or to Predict?", rich with references and quotes, which states: "Although not explicitly stated in the methodology
7.11.19 @8:43pm - (2/2) literature, applied statisticians instinctively sense that predicting and explaining are different." ... Going through the quotes and references in Hand's paper, I believe they confirm Shmueli's verdict: "the statistical literature lacks a thorough
7.11.19 @8:43pm - (3/3) discussion of the many differences that arise in the process of modeling for an explanatory versus a predictive goal." One may also note that, adding to a "thorough" discussion, the Ladder of Causation provides "formal distinctions" between those differences. #Bookofwhy

7.11.19 @4:49pm - (Replying to @namalhotra @matt_blackwell and 18 others) Whence do you think comes the urge to condition, despite "Didn't we know this already". Is there some textbook guideline that makes good people forget what they know? #Bookofwhy

7.11.19 @3:53pm - (Replying to @Scientific_Bird @elizpingree @BridgetPhetasy) You will not regret it. PRIMER is ideal, especially to Inquisitive-minded Birds.

7.11.19 @3:47pm - (Replying to @cutearguments @rlmcelreath) I was not aware of "Virtual History" when writing #Bookofwhy. Thanks. Gelman, on the other hand, is wrong; Rubin denies any meaning to "undoing past events". That's why he replaced "counterfactuals" with "potential outcomes".

7.11.19 @3:35pm - (Replying to @namalhotra @matt_blackwell and 18 others) This is quite shocking: "Overall, we find that 46.7% of the experimental studies published in APSR, AJPS, and JOP from 2012 to 2014 engaged in posttreatment conditioning (35 of 75 studies). " Something to keep in mind in scrutiniaing "real-life" experimental studies. #Bookofwhy

7.11.19 @6:48am - (Replying to @cdsamii @ingorohlfing and 14 others) What are the cultural roots of path models in political science? Duncan and LISREL? or some other source? When and how did PO enter into this field, as we can see in the works of King and Imai?

7.11.19 @3:56am - (Replying to @nasim_rahaman @MPI_IS) Gee, I recognize the books behind me, but not the Balcony. I hope it is not a "cross world" illusion. #Bookofwhy

7.11.19 @3:51am - For the physicists among us, especially those fascinated by quantum mechanics and Bell Inequality, DAGs appear to enlightened the conversation when we allow communication among observers, see

7.11.19 @1:34am - (Replying to @ingorohlfing @thosjleeper and 13 others) This is a good paper, thanks for posting. DAG's are used here not merely to convey assumptions, but also to visualize violations of assumptions and how these correspond to PO expressions. I can't imagine any paper on racial discrimination skipping DAGs in those roles #Bookofwhy

7.11.19 @1:11am - (Replying to @yskout @Jabaluck and 11 others) Robustness comes from making assumptions in the language of stored knowledge. And logical equivalence is not "modeling equivalence. Ask an economist to tell you if HIS latest model has testable implications. Can he do it? Sure! But compare his effort to students of DAGs#Bookofwhy

7.11.19 @12:49am - (Replying to @thosjleeper @PHuenermund and 12 others) I am an optimist. I imagine 18K out of our 19.1K followers to be silent observers today and vicious rebels tomorrow; they are not dumb. Yes, they have to worry about tenure, publications and other suppressants of academia, but they are no dumb. They see the potentials. #Bookofwhy

7.11.19 @12:31am - (Replying to @thosjleeper @PHuenermund and 12 others) I believe Imai and his students are recent converts and, in social science, we have Morgan, Winship and Elwert. #Bookofwhy

7.11.19 @12:21am - (Replying to @JDHaltigan @WiringTheBrain @PeterJungX) Agree. Mediation analysis is one of those areas where regression culture is deeply entrenched, and you can still find PhD theses written in stone-age style. I noted it in pp.324-5 of #Bookofwhy,

7.11.19 @12:12am - (Replying to @WiringTheBrain @PeterJungX) I am not sure to what extent your concern is valid today, 30 years after the Causal Revolution. I know there are pockets of regression analysts who refuse to elevate themselves from statistical thinking, but most of those in my circle are aware of ANCOVA's dangers #Bookofwhy

7.10.19 @11:05pm - (1/4) Happy Anniversary! About a year ago, I started Twitting and posted this: 6.27.18 - Hi everybody, the intense discussion over The Book of Why drove me to add my two cents. I will not be able to comment on every tweet, but I will try to squeak where it makes a difference....
7.10.19 @11:05pm - (2/4) A year later, I can hardly believe the 2,300 Tweets behind me, 19.1K followers, and a pleasant sense of comradeship with the many inquisitive minds that have been helping me demystify the science of cause and effect. I have benefited immensely from seeing how causality is
7.10.19 @11:05pm - (3/4) bouncing from your angle and how it should be presented to improve on past faults. To celebrate our anniversary, I am providing a link to a search-able file with all our conversations (my voice) sorted chronologically ... Today I re-read selected sections
7.10.19 @11:05pm - (4/4) and, believe me, there are quite a few non-trivial nuggets out there that should be retrieved, re-weaponized and reused. Finally, access to individual chapters of PRIMER is now available by clicking here Cheers, #Bookofwhy.

7.10.19 @6:38pm - (Replying to @MatthewMOConnel @GoldsteinBrooke @TuckerCarlson) We are shifting from defense to attack and, with your help, will make Zionophobia the ugliest word in town. Use it!

7.10.19 @5:21pm - (1/ ) (Replying to @fuzzydunlop123 @autoregress and 11 others) Economics? "Open to new methods"? The last open-minded economist I met was Hal White (1950-2012). When he passed away, his former students could no longer publish in top journals, and had to revert back to old methods (my interpretation). Now observe how hard Heckman and Pinto
7.10.19 @5:26pm - (2/ ) (Replying to @yudapearl @fuzzydunlop123 and 12 others) labor to preserve old methods ; pages after pages of derivations just to refrain from using d-separation. "Open to new methods"? Let's see how long it will take to Editors of top Econ Journal to invite ONE review paper on what graphical models can offer
7.10.19 @5:41pm - (3/ ) (Replying to @yudapearl @fuzzydunlop123 and 12 others) econometric researchers. This is 2019, almost 40 years after graphical models came into being and impacted almost every data-driven discipline, except economics. It's hard to believe, I agree, and surely our silent rebellious students will not call it "open minded" #Bookofwhy

7.10.19 @5:14pm - (Replying to @autoregress @fuzzydunlop123 and 11 others) Having been burned before by ungrounded theory is no excuse for refusing an eye-glass that helps you navigate your OWN theory.

7.10.19 @4:36pm - (1/ ) (Replying to @Jabaluck @yskout and 11 others) I agree, but would phrase it a bit differently. Any economist NOT familiar with DAGs would rejoice knowing that his/her most intimate chunks of economic knowledge can now be expressed in a scientifically prudent language, uncontaminated by parametric or statistical baggage
7.10.19 @4:51pm - (2/ ) (Replying to @yudapearl @Jabaluck and 12 others) ready to be submitted for mathematical or algorithmic analysis, in which each step is meaningful, that is, scrutinizable by the knowledge-providing economist, and which readily delivers answers to questions that otherwise take hours to answer. How about this phrase? #Bookofwhy

7.10.19 @3:36pm - (1/ ) (Replying to @Jabaluck @yskout and 11 others) 1/ Allow me to use your favorite: "You are missing my point". The key point that I am trying to make is that there is such a thing as "human representation of knowledge" and it has a cognitive library of primitive relationships from which more complex relationships are composed.
7.10.19 @3:44pm - (2/ ) (Replying to @yudapearl @Jabaluck and 12 others) The compound relation "prehospital behavior that might independently effect outcomes", is composed of many primitives relationships, each of the type "X directly affect Y." DAGs tap those primitive relationshiop directly, hence reliably. Compound relations require mental
7.10.19 @3:52pm - (3/ ) (Replying to @yudapearl @Jabaluck and 12 others) construction effort and are more vulnerable therefore to human error. This is the key to the clarity of DAGs, not "familiarity". "DAGs go to where knowledge resides" I once said. Any discussion of DAGs should start with this key observation; the rest are corollaries #Bookofwhy

7.10.19 @2:54pm - (1/2) (Replying to @Jabaluck @yskout and 11 others) 1/2 @Jabaluck Your resistance to DAG betrays your cultural upbringing (Rubin? Angrist? Imbens?) and refutes your own words: "We control for differences in prehospital behavior that might independently effect outcomes." Anyone who can judge if a difference "INDEPENDENTLY AFFECT
7.10.19 @3:04pm - (2/2) (Replying to @yudapearl @Jabaluck and 12 others) THINGS" can surely judge if "one variable directly affects another". The latter type of judgments is all that is required for constructing a DAG, hence it could not be as "incredibly challenging" as you describe it. This is universally true for any representation of knowledge
7.10.19 @3:11pm - (2/4) (Replying to @yudapearl @Jabaluck and 12 others) 2/4 and it does NOT depend "on whether people are familiar with the terminology." To semi-prove my point, notice that your recipe "We control for differences in prehospital behavior that might independently effect outcomes" is deficient, for it misses controls for variables that 7.10.19 @3:30pm - (4/4) (Replying to @yudapearl @Jabaluck and 12 others) 4/4 affect treatment, not outcome. So, it is hardly the case that "The language economists use seems efficient and unambiguous here." The language is ridden with ambiguities, which calls into question the credibility of key judgments issued by DAG-avoiding economists. #Bookofwhy

7.10.19 @2:13pm - (1/2) (Replying to @PeterJungX @WiringTheBrain) Oh, how I wish to see responses of statisticians to this question! How they interpret the words "works", "control for" "confounders" etc. How their answers vary depending on what rival camp they declare allegiance to, and more. Can we get a summary? My answer: Yes, if you are
7.10.19 @2:26pm - (2/2) (Replying to @yudapearl @PeterJungX @WiringTheBrain) versed in causal inference (CI) and its eye-glasses - graphical models. No, if you are a mainstream statistician, believing that "confounding" is a statistical notion. For a more detailed answer, see "confounding bias" pp. 53-60 of PRIMER #Bookofwhy

7.10.19 @5:12am - (Replying to @EpiSconroy @EpiEllie) Stat theory include probability theory, regression analysis, hypothesis testing, confidence intervals, etc. all are theories of the joint distribution functions that govern the observed data. #Bookofwhy

7.10.19 @3:58am - (1/ ) (Replying to @Jabaluck @autoregress and 10 others) I did not say I'll never convince you. I said: the reason I am spending time on tweeter is not in hope of convincing you or Angrist. I do it to empower the curious yet silent rebels among econ. students, what seems like an easier task. You (not sure about Angrist) will be
7.10.19 @4:15am - (2/ ) (Replying to @yudapearl @Jabaluck and 11 others) convinced (I hope) when one of these students asks in class: "Why can't we test the exclusion restriction by checking if E(Y|x, z) depends on z?" which will make you sorry for not teaching d-separation. You asked: what I have learned from economists, a question bothered me a
7.10.19 @4:36am - (3/ ) (Replying to @yudapearl @Jabaluck and 11 others) a lot, because I was hoping to tell my students: Economists have developed methods of solving problem A, R and T. But, aside from LATE, I have hard time giving A, R and T content that I can easily describe and comprehend. When I ask colleagues they send me to fancy articles
7.10.19 @5:03am - (4/ ) (Replying to @yudapearl @Jabaluck and 11 others) without telling me what nuggets of wisdom I can expect to find there if I try really hard. I made this plea on Twitter with not much success. But I have not given up; I know those nuggets exist and are waiting to be excavated. Perhaps by tomorrow's rebels #Bookofwhy

7.9.19 @10:56pm - (Replying to @yudapearl @EpiSconroy @EpiEllie) a carrier of scientific assumptions about the world outside the data (say populations, or individuals reactions to exposures) and should be used to exemplify ideas about scientific methods, hypotheses, evidence, predictions, abduction, inductions, etc etc..

7.9.19 @10:51pm - (Replying to @EpiSconroy @EpiEllie) The theories we learn in psych and stat are different indeed from the epi conception of a "theory". The first, because it was mainly verbal, the second because it was about the data, not the process generating the data. A DAG is a good embodiment of what we mean by a "theory" ,ie

7.9.19 @10:01pm - (1/ ) (Replying to @eliasbareinboim) As a strong advocate of "reality first, algorithm second" I should note that the level at which we model reality is sometime chosen to enable an algorithm. For example, an inference engine may issue the output: Sorry, your model does not allow for the identification of query Q
7.9.19 @10:05pm - (2/ ) (Replying to @yudapearl @eliasbareinboim) However, if you can only think of a variable Z that lies on the arrow X---->Y and satisfy additional properties, I would be able to identify Q using Algorithm-1. Likewise, if you can only think of W that...using Algorithm-2. Thus, the model is refined as we go along.#Bookofwhy

7.9.19 @3:41pm - (1/2) (Replying to @pablogerbas @Jabaluck and 9 others) I think you should get this published someplace, for the sake of people really interested in this applied research, so that they can see the layout of the problem clearly, and discuss substantive issues if any. Alternatively, you can publish it as an educational device to
7.9.19 @3:56pm - (2/2) (Replying to @yudapearl @pablogerbas and 10 others) enlighten X-econs with the way CI folks think about a problem, w/o dismissive calls for "real life problems". Or, perhaps a joint paper by all tweeting discussants. It will turn into a classics, perhaps even an underground "bubble-burster." I'll support it fully. #Bookofwhy

7.9.19 @3:30pm - (Replying to @autoregress @PHuenermund and 10 others) Before Galileo pointed his telescope towards the moon, he tried it on a tree, 2 km away, and saw nothing new, just the familiar old tree, but smiling in freshness. This story is fiction, and I am no Galileo, but DAGs are the eyeglasses of CI, no devils behind them. #Bookofway

7.9.19 @1:17pm - (1/2) (Replying to @autoregress @PHuenermund and 10 others) Sorry, I did not say "incapable". I curiously asked "what % of students can solve it?" Why? Because one can read "Harmless Economics" and "Mastering Metrics" 10 times over and find no clue on how to solve it. Moreover, I am sure that there are many secret rebels among those
7.9.19 @1:30pm - (2/2) (Replying to @yudapearl @autoregress and 11 others) students and readers, you may be one of them, who could not sit still seeing CI advancing to new heights and continue to act as if these advances have no bearing on economic problems. So, I am not surprised that % is rising and will continue to rise. I am tweeting here
7.9.19 @1:46pm - (3/3) (Replying to @yudapearl @autoregress and 11 others) not to convince Angrist or Jason, but to empower their curious and capable students to see through the X-Eco-bubble. And, yes, I am confident, very confident, that the bubble will burst as soon as one of them tells the others: Hey, look at this microscope! #Bookofwhy

7.9.19 @1:03pm - (Replying to @Jabaluck @PHuenermund and 9 others) You are again distorting my words, perhaps for realizing their truth. I never uttered the word "inferior". I said "suspect, for lack of ground truth". A doctor who insists on operating on patients w/o studying anatomy may be a great surgeon, but still highly suspect. #Bookofwhy

7.9.19 @6:30am - (Replying to @stephensenn @omaclaren @eliasbareinboim) Let's forget then the iid assumption and estimate E[Y|X=x] by OLS regression. Do I need to know about block design? I just randomized people to treatment and control by a fair coin. Where will I go wrong? #Bookofwhy

7.9.19 @6:10am - (Replying to @stephensenn @omaclaren @eliasbareinboim) Consider: (1) I randomize a treatment X and record data in the form of x,y pairs. (2) I forget that these pairs came from RCT and imagine that they are iid samples from some distribution P(x,y) (3) I estimate E(Y|X=x) under this illusion. Will the illusion hurt me? #Bookofwhy

7.9.19 @5:08am - A new explanation-seeking paper that hit my screen:

7.9.19 @4:27am - (1/2) (Replying to @omaclaren @eliasbareinboim @stephensenn) Yes, now that we find one reader thinking that the complete version makes a difference, we will try to include it in the upcoming paperback version. I personally think that it was his "reduction of data" mantra that defined 20th Cent. statistics agenda. Recall that Fisher
7.9.19 @4:37am - (2/3) (Replying to @yudapearl @omaclaren and 2 others) did not have notation for "causal effects". Even in the context of RCT, his concern was the reduction of data obtained from a randomized trial, not to assure correctness or unbiasedness. When he tried that (in mediation context), he blundered (ref = Rubin), #Bookofwhy. He
7.9.19 @4:44am - (3/3) (Replying to @yudapearl @omaclaren and 2 others) definitely did not have the concept of "causal assumption", which is essential for every task of modern "causal inference." Going from RCT-data to "causal effect" is indeed a matter of data reduction, the causal part is already prepared for you in the design. #Bookofwhy

7.9.19 @1:54am - Thanks you @PHuenermund for reading a "real life" study, and showing us that it is made up of the same biological tissues as "toy problems", only more of them. #Bookofwhy

7.9.19 @12:20am - (1/ ) (Replying to @Jabaluck @fuzzydunlop123 and 7 others) Please do not distort my words. You make it very unpleasant to interact with you when you do so. In our exchange I did not mention "economists" but X-econs, namely, model-avoiding economists of the quasi-experimental school. Nor have I mentioned DAGs, I spoke "models", which
7.9.19 @12:29am - (2/ ) (Replying to @yudapearl @Jabaluck and 8 others) includes structural economics. Finally, I never said: "economist do things wrong". I said there is no way of knowing if X-econs do things wrong, in the absence of ground truth, but it sounds very funny when a whole field prides itself on solving huge "real life" problems, but
7.9.19 @1:07am - (3/ ) (Replying to @yudapearl @Jabaluck and 8 others) ONLY huge "real life" problems, not "toy problems" which have ground truth and where everyone can see if you solved it correctly or not. Such problems are avoided like a plague, labeled "toy", "made-up" and worse, but never discussed in good company. I said "it sounds funny"
7.9.19 @1:14am - (4/ ) (Replying to @yudapearl @Jabaluck and 8 others) not in the hope of convincing you to take a step back and see how funny it is, but in the hope of confirming the feelings of hundreds, perhaps thousands of econ. students who I know are listening silently to this twitter exchange, perhaps after reading #Bookofwhy in hiding,
7.9.19 @1:22am - (5/ ) (Replying to @yudapearl @Jabaluck and 8 others) and asking themselves 3 times a day: Isn't funny that my professor can solve "real life" problems of such magnitude and importance, and he/she cant tell which scenario (scenario! not DAG) contains a legitimate IV? I tweet here to tell this silent student: You are not alone,
7.9.19 @1:38am - (6/ ) (Replying to @yudapearl @Jabaluck and 8 others) it is mighty funny, but it is going to change in a few years, and you can prepare yourself for the day when your field will change from "funny" to "well informed". Just make sure you spend the 2-3 hours it takes to acquire the art of causal modeling. I'll be there for you.

7.8.19 @11:50pm - (Replying to @Jabaluck @fuzzydunlop123 and 7 others) Show me one simple case where the exclusion requirement need not be justified. Now lets go to any "real life" problem and examine HOW it was justified, if at all. I cant find a model (not a DAG, a model) in the papers you ask me to read, so how can one tell if the reported
7.9.19 @12:03am - (2/ ) (Replying to @yudapearl @Jabaluck and 8 others) results are not biased substantially by violations of exclusion? You know that there is objective validity test to the results reported, and you are asking me to go through the numbers and show that they can do better with DAGs. The 4 scenarios we discussed tell us more about
7.9.19 @12:11am - (3/ ) (Replying to @yudapearl @Jabaluck and 8 others) problems with exclusion that 100-page "real life" study where those problems are not modeled (forget DAGs, by ANY model). If you think I could learn something from a "real life" article, please tell me what principle I can gain from it that I may not know already. #Bookofwhy

7.8.19 @11:09pm - (Replying to @Jabaluck @autoregress and 6 others) Retweeting: Sorry, these are not "toy problems"; they haunt each & every IV exercise, albeit suppressed by practitioners to the mercy of intuition, so as to escape "struggle". Each one represents 10,000 "real world" problems in which "exclusion" is/was justified. #Bookofwhy

7.8.19 @9:28pm - (Replying to @Jabaluck @autoregress and 6 others) Wrong. I answered it head on: "econs use too many in lieu of other approaches". I even proposed an explanation for this imbalance: X-econs avoid models because they do not know how to handle them mathematically. Care to estimate of % of Angrist's students who know how?#Bookofwhy

7.8.19 @9:11pm - (Replying to @Jabaluck @autoregress and 6 others) Sorry @Jabaluck, these are not "toy problems", because they haunt each & every IV exercise, albeit suppressed by practitioners to the mercy of intuition, so as to escape "struggle". Each one represents 10,000 "real world" problems in which "exclusion" is/was justified. #Bookofwhy

7.8.19 @8:59pm - (Replying to @omaclaren @eliasbareinboim @stephensenn) I wish someone would plot the frequency of the words "cause" or "causal" vs. time, from 1900 to 2019 and tell us, straight face, "No! There was no causal revolution in 1990s". (There were 13 such papers in JSM 2003) Why are u resisting the idea of a paradigm shift? #Bookofwhy

7.8.19 @8:29pm - (Replying to @omaclaren @eliasbareinboim @stephensenn) Statistics as a field is defined in two arenas. (1) What its leaders say and do. (2) What its textbook say and do. For (1) I read the presidential addresses in the past 20 years. For (2) I look at the index. #Bookofwhy

7.8.19 @8:24pm - (Replying to @omaclaren @eliasbareinboim @stephensenn) I dont think #Bookofwhy states that 1)Statistics doesn't deal with causality. It quotes Fisher's definition: "Statistics is summarizing data," and it decries the vacuum in stat textbook, and it places Fisher's DoE precisely where it belongs in history of CI. What would you add?

7.8.19 @4:59pm - (1/ ) @autoregress I am not an extremist, so I am not worried about econs missing out on complex DAGs. I am concerned about econs missing out on their simple IV models like the ones discussed here: ... 4 scenarios, 4 variables and we havn't found an X-eco who
7.8.19 @5:28pm - (2/ ) (Replying to @yudapearl @autoregress and 6 others) 2/ was able to tell us which of the 4 scenarios has a legitimate IV. Bad sample? Perhaps. So let me ask: How many of Angrist's @metrics52 students can do it? And if this does not jolt X-econs to do some honest soul-searching, what will? Their credibility is at stakes. #Bookofwhy

7.8.19 @1:37pm - (Replying to @Jabaluck @autoregress and 5 others) @Jabaluck I believe you misunderstood me. I have not used the words "back-door" in any of my tweets (to u). I used "sources of variation" which was your term. My main point is: X-Eco refrain from using a model for two reasons: (1) The mistrust the assumptions behind the model and
7.8.19 @2:00pm - (2/ ) (Replying to @yudapearl @Jabaluck and 6 others) (2) They mistrust models period, even when they trust a set of assumptions. I am only concerned about reason (2). If I am wrong, I will convert back to X-Eco. How can you prove me wrong? Show me one X-Eco paper that explicates and combines assumptions formally #Bookofwhy

7.8.19 @1:32am - (Replying to @vijayant_k @Canadian_JACD and 2 others) We are concerned with the effects of BP not with its causes. If it has an effect on cardiac stress we should be allowed to include the arrow BP--->Cardiac Stress in our model, and not to ask for supreme court permission.

7.8.19 @12:57am - (1/ ) (Replying to @mc_hankins @stephensenn @eliasbareinboim) 1/ Since we started working on external validity, transportability and data fusion in 2010 (eg, ) we have been hearing the whisper: "This sounds like meta-analysis." Yes we are yet to find a meta-analyst expert who can to tell us how to handle the simplest
7.8.19 @1:07am - (2/ ) (Replying to @yudapearl @mc_hankins and 2 others) example. (and I went to the very top). Why? Because we can show that the answer depends on the causal relationships between X1,X2,Y and other factors in the problem. Meta-analysis is a statistical pooling method that is oblivious to those relationships. Therefore, I take the
7.8.19 @1:17am - (3/ ) (Replying to @yudapearl @mc_hankins and 2 others) liberty to assume that meta-analytic efforts are orthogonal to the kind of problems we are trying to solve. However, I will immediately change my mind if I find a Meta-Analyst who solves the examples of or Fig. 1 of #Bookofwhy

7.7.19 @11:02pm - (Replying to @yudapearl @Jabaluck and 6 others) 4/ explicitly what I understand about each of those sources separately, and try to combine those understandings mathematically to decide if Z is a good IV. I was tempted to do (2) but, then, I realized to my horror that I am re-committing DAG-heresy. What shall I do? #Bookofwhy

7.7.19 @10:36pm - (1/ ) (Replying to @Jabaluck @MariaGlymour and 5 others) @Jabaluck your arguments are so persuasive that I decided to quit DAGs and convert to X-Eco (short for Experimental Economist). The moment of conversion was truly enlightening and, as if by divine revelation, I began to "understand the sources of variations" of variables,
7.7.19 @10:44pm - (2/ ) (Replying to @yudapearl @Jabaluck and 6 others) something DAG folks never understand. Fired by enlightenment, I chose Z as my potential IV, and I understood the "source of variations" of many other variables, X,Y,Z,W,S, T, V, all related to Z, to X and to Y. I truly understood those "sources of variations, and I felt elated
7.7.19 @10:51pm - (3/ ) (Replying to @yudapearl @Jabaluck and 6 others) Now I had to decide whether Z, which looked like a promising IV before, is still a good IV given what I understood about all the other background variables. I had two options: (1) Decide yes or no based on my understanding of all those sources of variations or (2) articulate

7.7.19 @10:22pm - (1/2) Sailors and passengers on this voyage of CI research should be interested in these new results that just reached my screen, , by @eliasbareinboim & team. Suppose we randomize treatments X1 and X2 in two separate studies, can we estimate the causal
7.7.19 @10:22pm - (2/2) effect of their conjunction (X1=x1, X2=x2) ?? I see dozens of pharmaceutical companies rushing to join our voyage. #Bookofwhy

7.7.19 @9:39pm - (Replying to @eliasbareinboim @EpiEllie and 2 others) I am willing to adopt the new construction method if it helps science. But we are still in need of a rule on when to include an arrow BP--->Y and when not to, assuming that we sear never to ask for the effect of BP on anything. What should we think about to decide? #Bookofwhy

7.7.19 @8:56pm - (1/ ) This is a brilliant idea, to use the causal hierarchy in "critical thinking" classes: Given an English expression, classify it into rung-1, 2 or 3. Some of the top researchers of our time should take it. I remember when our kids were young we got them a game called "propaganda"
7.7.19 @8:56pm - (2/ ) Each card had a false argument and players had to classify it into one of several types of falsehood. Great!! As to a simpler/gentler version for high schoolers? The #bookofwhy is all we have, but it should be fun to do one, with a creative cartoonist. Great!!

7.7.19 @5:59pm - (Replying to @yudapearl @raymondshpeley @f2harrell) certain aspect of the distribution, eg P(label|features), which is no more of model than say the estimated regression coefficient R, fitted to a cloud of samples. Would you call R a model? Does it carry any assumptions? Only if you havn't seen any of the data. #Bookofwhy

7.7.19 @5:51pm - (Replying to @raymondshpeley @f2harrell) Mathematically, models are carriers of assumptions. Stat models carry assumptions about the distribution, eg normality, binomial,.. and causal models carry assumptions about causal relationships, eg. who listens to whom. A learned neural net provides efficient representation to a

7.7.19 @4:50pm - Answer to What are the differences between the "experimentalists" and "structuralists" approaches to econometrics? by Judea Pearl

7.7.19 @2:57pm - (Replying to @vijayant_k @Canadian_JACD and 2 others) According to all experts I talk to, BP is not just an indicator of some risk factor, but it actually causing bad things, eg. heart stress. We do not want to go to extremes and claim that we are only seeing the display on the measuring device, not the BP itself. #Bookofwhy

7.7.19 @12:44pm - (Replying to @f2harrell @raymondshpeley) A point which I should have added (based on @f2harrell tweet) is that whereas statistics can give us an (estimated) distribution, DL gives us one aspect of the distribution, eg P(label|features). DL folks may argue that, in principle, they can learn the P as a function X-->(0,1)

7.7.19 @12:33pm - (Replying to @MariaGlymour @PHuenermund and 5 others) Who sold us "empirically validated" ointment? When I say a "valid instrument" I mean one that satisfies the the IV requirement according to the model that was proposed. Plus, that model may have testable implications.

7.7.19 @5:48am - (1/2) (Replying to @PHuenermund @Jabaluck and 5 others) I guess what the experimentalists are arguing is as follows: If you are committed NOT to put down any model on paper, and to work purely by intuition, then it is easier to start by asking yourself: "Is there an exogenous variable around that is somehow related to X and Y"? as
7.7.19 @5:58am - (1/3) (Replying to @yudapearl @PHuenermund and 6 others) as opposed to asking yourself: "Is there a way to control ALL confounders of X and Y?". Next comes the handling of "somehow related". If you are still committed to model-blind thinking, you will have to intuit handwaving about the exclusion restriction, which experimentalists
7.7.19 @6:07am - (3/4) (Replying to @yudapearl @PHuenermund and 6 others) are willing to stomach given the ease of having found SOME relevant exogenous variable in the mental forest. What they forget is that there is a middle ground: Find your favorite exogenous variable, model what you know about the relationships with X,Y and other factors, then
7.7.19 @6:19am - (4/5) (Replying to @yudapearl @PHuenermund and 6 others) use your modeling tools to decide if you have a valid IV, if not, can you repair it, can you find another, can you test your assumptions etc etc, all the nice thing that models give you, rather than remaining in the intuitive world. The trade off will be settled only when
7.7.19 @6:28am - (5/5) (Replying to @yudapearl @PHuenermund and 6 others) experimentalists agree to learn how to model problems with 5-10 variables, and see for themselves what they have missed by resisting it. #Bookofwhy

7.7.19 @4:51am - (Replying to @pentagoniac) Evidently I violated some cosmic rule of Quora protocol. No idea what was violated nor who vowed for my innocence.

7.7.19 @4:16am - Victory! My appeal was accepted. My answer on "econometrics, statistics, and machine learning" was reinstated on Quora, and my honor restored. I bet they were impressed by the clean life I live. #Bookofwhy

7.6.19 @7:51pm - (Replying to @mattshomepage) It is a mystery which I tried to answer in chap. 10 of #Bookofwhy. Millions of years of playful experiments with bows and arrows are encapsulated in our culture, our language, our elderly wisdom, our books etc. The practical question is: how to represent it and how to exploit it.

7.6.19 @7:28pm - (Replying to @y2silence @PWGTennant and 3 others) No one knows what they say. All we hear is: If A is not randomized (or randomizable) then "it all depends".

7.6.19 @4:49pm - (Replying to @imleslahdin) Answer deleted!! Thanks for telling me. I guess the vice-squad just found out about me. I appealed. Lets see how they handle "appealers".

7.6.19 @4:18pm - (1/2) (Replying to @AngeloDalli @EpiEllie and 2 others) An arrow BP--->Y has two interpretations: (1) Y listens to and responds to changes in BP. (2) Manipulating BP changes Y. (1) implies (2), but some may argue that (2) is all we can observe, hence it is "scientific" while (1) is "meta-physics". Now, since BP is not manipulable,
7.6.19 @4:26pm - (2/2) (Replying to @yudapearl @AngeloDalli and 3 others) adding an arrow BP--->Y to your model makes you suspect of membership in the "meta-physics" camp, which carries harsh consequences. This is why the BP issue is foundational, and that is why we see such hesitations and "it all depends" from the manipulationist camp. #Bookofwhy

7.6.19 @4:04pm - (Replying to @f2harrell) Do you think statistics offers more than "taking us from samples to properties of distribution functions" ?? If you do, then I confess to underplaying the role of #statistics. But did I? #Bookofwhy

7.6.19 @2:16pm - (Replying to @neuro_data @danilobzdok) Progress in plain geometry was very very advanced before someone said: lets add a 3rd dimension. If we want to remind people that the world is 3-dimensional it is helpful, I think, to label 3D-geometry "advanced".

7.6.19 @2:07pm - (Replying to @yudapearl @JaapAbbring) Here is a brilliant idea. As co-Editor of the Journal of Causal Inference I will invite you to tell our readers how they can benefit from tools developed in Econ. and you will reciprocate by convincing a mainstream Econ. journal to do likewise. Do we have a deal? #Bookofwhy

7.6.19 @1:59pm - (Replying to @yudapearl @JaapAbbring) involves graphical models, do-calculus, counterfactual logic etc. which do not rule out tools produced in econ. to deal with part (1). I see wide awakening on both sides to examine and evaluate each other tools. But it is only a Twitter-awakening, not in journals or NBER. Wait!!

7.6.19 @1:49pm - (Replying to @JaapAbbring) The words "relatively simple" are Imben's words, with which he justifies why he can avoid graphical models. See ... SCM has two parts: (1) a model of reality and (2) tools for dealing with (1). The first is identical to structural economics. The second

7.6.19 @1:29pm - (Replying to @danilobzdok @f2harrell) Neural network is a man-made artifact, crafted to capture some aspects of the data, say the relationship between symptoms & disease. By a "model" we mean a picture of reality, from which the data emerged. e.g. X--->Y and X<---Y are two different models that may yield same NN.

7.6.19 @1:21pm - (Replying to @f2harrell) I agree. Most neural networks do not involve models of the data-generation process. Agree. I was afraid you have found points of disagreement.

7.6.19 @2:36am - (Replying to @robertwplatt @Canadian_JACD and 3 others) Would adding or not adding an arrow from BP change whatever you are interested in doing? If it does, Eureka! We just discovered the scientific meaning of an arrow going out of nonmanipulable variable. Now we ask: How would you decide whether to add or not to add? #Bookofwhy

7.6.19 @1:49am - (1/2) (Replying to @JaapAbbring) I do not recall describing econometric models as "relatively simple," especially not structural econometrics, which can be viewed as a subset of SCM, lacking graphical tools and do-calculus. I may have used the phrase "relatively simple" to describe the kind of problems
7.6.19 @1:58am - (2/2) (Replying to @yudapearl @JaapAbbring) that can be handled by IV methods Angrist style, where no model is laid down for analysis and scrutiny, or PO methods Imbens-Rubin style, where ignorability assumptions beg to be discerned. Structural models + graphical tools is a winning combination #Bookofwhy

7.5.19 @11:49pm - (Replying to @davidmanheim @kareem_carr @EpiEllie) True, basic econometrics was meant to do CI, but look at the tools they are still using in the 21st century, not even realizing what tools are missing from the Econ-Echo-Chamber. As to DNN's etc. they were "advanced" five years ago; times they are achangin. #Bookofwhy @causalinf

7.5.19 @10:32pm - (Replying to @dkipb @EpiEllie and 3 others) I have not received comments on "Q as a limit" or on ... (which is a beautiful paper, no bias), so we do not know if it helped resolve the hesitations. But "first pass" is a cop out; this is a foundational issue: Listening vs. manipulating. #Bookofwhy

7.5.19 @10:00pm - (Replying to @PWGTennant @EpiEllie and 2 others) God forbid. What I can or cannot obtain has nothing to do with the meaning of an arrow coming out of BP, which simply says: someone is listening and responding to BP. I have not heard Ellie&Miguel say "yes", perhaps b/c the notion of "Well-defined" is not well defined? #Bookofwhy

7.5.19 @9:51pm - (Replying to @PWGTennant @EpiEllie and 2 others) Disagree. I believe the inconsistencies and hesitations of the "Not-Well-Defined" school have created more confusion than usefulness. All in the name of mimicking RCT's. #Bookofwhy

7.5.19 @9:44pm - (Replying to @PWGTennant @EpiEllie and 2 others) Identification should not mar questions of meaning. And would stay away for obesity because it is also ill-measured. So its better to focus on BP, which is well-measured and still debatable (at least in some circles). #Bookofwhy

7.5.19 @9:40pm - (Replying to @PWGTennant @EpiEllie and 2 others) I wish I could, like you, agree with both sides. But I still do not know what one side says about an arrow out of BP. When is it legit?

7.5.19 @9:33pm - (Replying to @melb4886 @EpiEllie and 2 others) By saying "because alcohol raises BP" you just admitted the presence of an arrow out of BP. Did you get a license from the guardians of Well-Definedness.? They are still debating if/when such an arrow violates scientific right and wrong. #Bookofwhy

7.5.19 @9:28pm - (Replying to @vijayant_k @Canadian_JACD and 2 others) But if BP is merely an indicator for some other risk, we should not see an arrow going out of BP. But in all DAGs presented here we did see this arrow. Can you check with the experts? Still, the fact that we ask this question means that drawing such arrow make a difference.

7.5.19 @9:10pm - (Replying to @vijayant_k) Last I heard from ML folks was they are all working on "combining CI and ML". I heard if even from people who have no idea what CI means (names withheld). So why would anyone object to be working on "advance ML"? #Bookofwhy

7.5.19 @9:03pm - (Replying to @s_monterohdz) "Advance" does not rule out "super-advance" in the future. I believe I did define the scope of SCM in my writing in "real phenomena", Each DAG you draw represent millions of "real phenomena" that fit the structure. #Bookofwhy

7.5.19 @8:56pm - (Replying to @ADAlthousePhD @EpiEllie) Thanks, I'll try to reply, instead of retweet.

7.5.19 @1:26pm - (Replying to @kareem_carr) Good point. I have been trying hard to have statisticians embrace CI and call it "advanced statistics" watch where they are. As to econometrics, intervention was the goal of even "elementary econometrics,", only the tools that are lacking. #Bookofwhy

7.5.19 @1:10pm - (Replying to @f2harrell) Let's hear ONE disagreement. We settled the previous ones, we can do same with this ONE.

7.5.19 @7:21am - Answer to What are the differences between econometrics, statistics, and machine learning? by Judea Pearl

7.5.19 @2:17am - (Replying to @EpiEllie) We are almost getting there. Quoting: "If.....then we can ask causal questions involving THAT INTERVENTION". But you did not say: "...causal questions involving Blood Pressure". Was it on purpose or by oversight? Moreover, would you then allow an arrow from BD? @Bookofwhy

7.5.19 @1:03am - (1/2) (Replying to @autoregress @EpiEllie) To interpret what @MariaGlymour wrote I would add: g-methods become valid "methods" only after you have a DAG to help you decide what variables to condition on, as in the back-door condition. Additionally, having or not having an IV is also a task dedidable by a DAG. Further,
7.5.19 @1:16am - (2/2) (Replying to @yudapearl @autoregress and 2 others) deciding whether a valid "g-method" exists (namely if identification can be done by regression" takes one glance over a DAG, it takes forever using ignorability assumptions as those used by Rubin, Imbens and Angrist. Lastly, I would not dismiss do-calculus #Bookofwhy

7.4.19 @11:07pm - (Replying to @EpiEllie) So, the answer is YES. We can include BP in the DAG, with an arrow going out of BP, since we do not require that BP be directly manipulable. It is enough that there are Well-defined interventions someplace in the model. Right? Can I tell this to my students? #Bookofwhy

7.4.19 @10:49pm - (Replying to @yudapearl @Canadian_JACD and 3 others) So, by all means, please post more and more DAGs with arrows emanating from BP. The more you post the less credible would their claim become, of the "new philosophy", that they are doing "experimental science" whereas you are doing "metaphysics". #Bookofwhy

7.4.19 @10:34pm - (Replying to @Canadian_JACD @EpiEllie @BL4PublicHealth) The "new philosophy" is the new school of "no causation w/o manipulation" that is brewing at Harvard (perhaps other places?) according to which you cannot talk about X causes Y unless X is manipulable. See @_MiguelHernan papers and for full view #Bookofwhy

7.4.19 @9:30pm - No No, please, continue. You are giving us straight answers: "blood pressure can be an important cause of things like the thickening of arteries". Please continue, because the very notion of "BP can be a cause" is under the danger of extinction, in the new philosophy #Bookofwhy

7.4.19 @9:18pm - I was waiting for this answer "the role of blood pressure in the causal question". It is the kind of answer @EpiEllie could not give, because it violates the dictates of RCT imitation. BP has no well-defined "role", only interventions have "roles". See

7.4.19 @9:07pm - (1/2) Not so easy. So the answer is YES. After thinking, weighting, assuming, understanding, specifying, etc. there might come a moment where you would add BP to the DAG. Now innocent me asks: why? Who needs this ill-defined entity there? Or, what can we do after adding it that
7.4.19 @9:07pm - (2/2) we couldn't do without, with all the thinking, weighting, assuming, understanding, specifying, etc. that gave us the license to add it? After all, if our research question is well-defined: "the effect of some drug", why do we need all this BP nuisance? Who cares? #Bookofwhy

7.4.19 @8:34pm - Not a simple question? OK, I will make it MUCH simpler: Is there any circumstance, after you do the thinking, thinking and more thinking, that you would add "blood-pressure" to your DAG? #Bookofwhy

7.4.19 @7:44pm - (Replying to @JadePinkSameera) It is a good question. And a very simple one too. And that's what makes it so hard to answer.

7.4.19 @7:22pm - "Are economists smarter than epidemiologists?" Our recent Twitter posts made me re-read the blog discussion we had in 2014: ... which I still find to be illuminating of the interplay between cultural and technical forces in scientific progress #Bookofwhy

7.4.19 @7:02pm - So, the simple answer to my innocent question is: NO. @EpiEllie and @causalinferenc and @_MiguelHernan would NOT include a variable "blood pressure" in a DAG before deciding how to weigh all the well-defined interventions on diet, drugs, exercise, etc, etc Am I right? #Bookofwhy

7.4.19 @6:29pm - (1/2) (Replying to @kareem_carr @jenniferdoleac and 2 others) @kareem_carr , you have infinite memory. You just confirmed my last Tweet. Many readers will resist my theory that an innocent curiosity of just two individuals can account for such profound differences between two disciplines. Many will seek differences in substance or
7.4.19 @6:39pm - (2/2) (Replying to @yudapearl @kareem_carr and 3 others) philosophy, or type of data etc etc. No way. It is as simple as your Tweet. Escaping from the echo-chambers of social bubbles is the strongest force that drives scientific progress. And it is becoming harder and harder in the age of internet and social media. #Bookofwhy

7.4.19 @6:15pm - (1/2) On the difference between Econ and Epi, I also said there are no substantive differences whatsoever between the two: ... The current differences in practice emerge from one fluke of history: Robins and Greenland were epidemiologists, not economists. Thus,
7.4.19 @6:15pm - (2/2) they followed curiosity and asked: what can DAG do for us? The rest is history. Why didn't economists follow their curiosity? Let others answer it because I do not want to spoil it for the many econs who ask this very question today. @causalinf , @jenniferdoleac #Bookofwhy

7.4.19 @5:31pm - All these complications need consideration, I agree. But let's ask an innocent question that comes before complications strike: "Can we assume that the variable "blood pressure" may appear in one of your DAGs and, when it appears, there is a arrow going out of it?"#Bookofwhy

7.4.19 @5:18pm - (Replying to @_gbmari) This self-fulfilling effect should be balanced against the spite-him effect of employees that are driven by the challenge to "prove him wrong". We agree I hope that both motivations demand causal models and cannot be accounted for by statistical considerations alone. #Bookofwhy

7.4.19 @4:07pm - Am I right to assume that the variable "blood pressure" may appear in one of your DAGs and, when it appears, there is a arrow going out of it, into other variables. Can we assume that much? #Bookofwhy

7.4.19 @2:02am - Phelps aside, do you think economists today are equipped to define+manage issues of "fairness" with their current tool set? #Bookofwhy

7.4.19 @1:23am - Your paper is an eye opener, glad you posted it. If it was not 1am I would have continued reading it, for it is written in a CI language and a well-structured style. The fact that it won @SIGMOD2019 best paper award may signify a new age for fairness folks. #fairness #Bookofwhy

7.4.19 @12:20am - I do not doubt that adequate causal defs of discrimination and fairness can be constructed by CI folks; they know causal models and counterfactual logic. I questioned the readiness of mainstream ML folks who are lacking those tools. Same goes for economists. #Bookofwhy

7.3.19 @10:35pm - (1/2) Yes, structural counterfactuals escape the torture chambers of the RCT's imitation-game. I think epidemiologists too are on their way to escape those chambers; how else can we interpret the arrows emanating from "blood-pressure" -- a well-measured yet
7.3.19 @10:35pm - (2/2) ill-manipulated variable that certainly has "effects" and often finds itself in DAGs drawn by epidemiologists? @EpiEllie @_MiguelHernan @Lester_Domes @MariaGlymour A puzzle. #Bookofwhy

7.3.19 @7:12pm - Thanks for re-posting, Brooke, and let our mantra for July be: "Make Zionophobia the ugliest word in town"

7.3.19 @7:03pm - For the next step, after #Bookofwhy, I will continue to recommend the PRIMER , , as long as it finds no match in clarity, examples and philosophy. Totally liberated from RCT's and other hangups, and essentially free. #Bookofwhy

7.3.19 @6:54pm - (1/3) Economists' "statistical discrimination," as it turns out, is both (1) the use of statistical associations to discriminate and (2) an attempt to define discrimination using statistical vocabulary alone. According to Phelps (1972), you discriminate whether you hire people by
7.3.19 @6:54pm - (2/3) education, race or zip code, as long as you base decisions on PREDICTED performance, rather than performance itself. So, all learning is "discriminatory" for it is based on past experience which PREDICTS, yet is not equal, the situation at hand. Conclusion: I was right
7.3.19 @6:54pm - (3/3) to suspect criteria entitled "statistical discrimination" and their ability to capture notions such as "fairness," in which causal relations play a major role. @JaapAbbring @steventberry #Bookofwhy

7.3.19 @6:24pm - (Replying to @swadhin_pradhan @geomblog) I dont recall giving such tutorial. Do you have a source?

7.3.19 @5:44pm - The counterfactual framework with which I am familiar (eg ) needs no interventions to have "effects". The paper you cite represents the "potential outcome" framework, a relic of an age when causation was forced-married to RCT (or imitations of). #Bookofwhy

7.3.19 @4:20pm - Statistics itself came from causal motivation for, surely, good predictions are essential for good decision (eg carry an umbrella). The missing ingredient is getting confidence to let these motivations out of the closet and articulate them mathematically #Bookofwhy

7.3.19 @3:31pm - I was under the impression that the issue of "manipulativity" was settled by explicating the non-manipulative aspects of "effects". Examples are: , . #Bookofwhy

7.3.19 @3:02pm - Footnote (1) in this paper ( ) confirms your point and explains why causal vocabulary is not at the "Frontiers" of Fairness research; it takes a generation to undo the statistical thinking that rules ML textbooks, classrooms and research practices.#Bookofwhy

7.3.19 @2:51pm - I cannot see how causal inference could "miss" injustices that statistical inference discovers, when the former represents reality and the latter a silhouette of reality, as projected in data. Example?? #Bookofwhy

7.3.19 @2:43pm - (Replying to @Adelaee) You made my day, Adelaee. The thought that your teenagers are reading my words compels me to write more, to make Zionophobia the ugliest word in town.

7.3.19 @1:48am - (Replying to @_onionesque @geomblog) I join you in this optimism. I have even blessed the pre-scientific hype for channeling good people to study causal and counterfactual models, then turn hype into science. It is happening, agree, because the tools for defining "fairness" correctly are available.#Bookofwhy

7.3.19 @12:48am - Thanks for correcting me on what economists mean by "statistical discrimination". I was not critical of the argument, but of the title, which alerted me to a possible new oxymoron, along with "probabilistic causality", "statistical confounding" "statistical mediation"..#Bookofwhy

7.3.19 @12:37am - (Replying to @NeuroStats) The first paper you cite is indeed the first I have seen on the right track. And I wish ML-folks will take notice. The second is an IV ACE-estimator, a rung-2 exercise, which cannot capture the counterfactual nature of "discrimination" (rung-3). #Bookofwhy

7.3.19 @12:19am - (1/3) If by "statistical discrimination" we mean the use of statistical associations in the decision, then it is perfectly harmonious with the causal definition of "discrimination" and I see nothing wrong with it. What makes me suspicious are attempts to DEFINE discrimination by
7.3.19 @12:19am - (2/3) statistical criteria. Note that in your example, causal considerations are unavoidable, for if the observed characteristics causally produce/prevent necessary skills, our notion of discrimination would change. This sensitivity to causal relations is absent from the fairness
7.3.19 @12:19am - (3/3) literature I have sampled thus far, and which seems to dominate recent discussions in ML. Note also that we now have the tools to define and manage criterion based on combined statistical+causal relations. BTW, the link you gave us is blocked. #Bookofwhy

7.2.19 @11:30pm - I've never dismissed an argument in which statistics and causality appear together. I am suspicious however of arguments that attempt to define inherently causal notions (eg discrimination) in terms of statistical vocabulary ALONE, void of causal relations. Wouldn't U? #Bookofwhy

7.2.19 @7:44pm - The title "statistical discrimination" worries me, because "discrimination" is a causal, not statistical notion. The words "from economics" makes me doubly worried, for reasons explained in #Bookofwhy. But I would love to hear how statisticians define "fairness". Truly curious.

7.2.19 @7:34pm - (Replying to @ahmaurya @alexdamour and 8 others) So how can we explain how FATE-motivated folks can study, write and speak FATE without applying some of the tools that CI is offering the ML community? For example I cant find the tools of Attribution Analysis and Causal Mediation invoked, Or am I missing them? #Bookofwhy

7.2.19 @5:12pm - (Replying to @eschisterman1 @AmJEpi) Congratulations!! Enrique.

7.2.19 @4:00 - (1/2) @marypcbuk @katecrawford @eliasbareinboim @benedictevans All definitions and examples of "fairness" that I have seen revolve around causal and counterfactual considerations, since they concern EFFECTs of policies on different segments of society. Thus, I venture to predict
7.2.19 @4:00 - (2/2) today's "fairness" hype will channel good people to study causal and counterfactual models, and help turn hype into scientific disciplines that define and algorithmitize the kind of fairness we wish AI systems to exhibit. There is virtue in pre-scientific hype. #Bookofwhy

7.2.19 @3:07pm - Agree. With two points of caution:(1) We need to study carefully the theoretical impediments to automated generation, to avoid falling for fake-gold, and (2) We need to study carefully what we can and cannot do with a causal model once we have it. The rest is just math #Bookofwhy

7.2.19 @6:38am - Thank you. I begin to understand. "Bias" is used here in the social sense, like discrimination. I am relieved. In our corner of the wood "bias" has technical meaning, standing for systematic deviation from expectation. eg confounding bias or "selection bias". Relieved #Bookofwhy

7.2.19 @6:26am - Interesting indeed, because I normally find people complaining of deep learning being opaque, here is one complaint: . Optical illusions are known to be mysterious. #Bookofwhy

7.2.19 @6:20am - I am not disputing the article, just expressing surprise at a term that I have not heard before. Can I conclude then that people who talk "AI Biases" mean the same thing as "limitations and biases of correlational ML." like those attributed to Rung-1 of the Ladder in #Bookofwhy?

7.2.19 @6:11am - Replying to @marypcbuk I couldn't pass the pay wall, but your myths are ML myths, which I can understand. I am still to understand what "AI bias" is which is not a myth. Are any of your 9 myths remediable by any of the 7 tools here:

7.2.19 @5:20am - (1/2) I did not realize that people call these problems "AI-bias". Thanks for bringing it to my attention and to the attention of my Twitter followers. Honestly, I have been swimming in the AI pool since the 1970's and never heard the term "AI bias" used before, especially not to
7.2.19 @5:20am - (2/2) describe problems that AI folks are about to solve in the near future. But, as they say in the Talmud: "Never too late to learn". Now, suppose I want to define the term on Twitter. Is every difficulty encountered by an AI program an "AI-bias"?? Which ones are not? #Bookofwhy

7.2.19 @4:36am - (Replying to @pentagoniac @benedictevans) I would love to see an example of a causal problem (say Simpson's paradox) that "*can* be corrected for without causal models. #Bookofwhy

7.2.19 @4:33am - (Replying to @benedictevans) But by saying "AI-bias" you give people the impression that such biases are inherit to AI, namely, permanently irredeemable by any AI program. Do we want to give general audience this impression?

7.2.19 @2:45am - I read @benedictevans on "AI-bias" and I still do not know what he means by "AI-bias", why not call it "ML-bias" or "curve-fitting bias" and how those biases can be avoided w/o attending to causal models as outlined here or here #Bookofwhy? A ML puzzle.

7.2.19 @2:22am - (Replying to @zacharylipton) I dont exactly understand what the time-management problem is. My students only worked on things that other people said are impossible, so mentoring was part of speaking to colleagues. I am not finished though, some colleagues still say they can do astronomy without telescopes.

7.2.19 @1:59am - (1/2) Nice and concise summary of #Bookofwhy. If I were forced to make a critical comment, it would be the way it tries to stimulates readers interest by pointing to existing interest. What if no ML conference had any session on causation, would that make the ideas of #Bookofwhy
7.2.19 @1:59am - (2/2) less compelling for enlightened ML folks trying to build intelligent systems? #Bookofwhy was written to change, not to follow habits. I hope it does.

7.2.19 @12:05am - (Replying to @mimblewabe) I considered changing "do" to avoid criticism like "You cant do this" or "some 'do's change everything". But I decided against it b/c there was no sub. We, Sapiens, do not distinguish between reality and models of reality. Even Abraham asked "what if there were 50...". #Bookofwhy

7.1.19 @11:56pm - Thanks for adding your smiles to this memorable event.

7.1.19 @11:52pm - For something more mathy, I will continue to recommend the PRIMER until something better shows up in the jewelry store. Note that it is now (essentially) open-accessed. #Bookofwhy

7.1.19 @11:22pm - (Replying to @causalinf @dlmillimet and 2 others) Astronomy will never be the same.

7.1.19 @11:19pm - (Replying to @IndexLlc @IARPAnews) I saw the abstracts and pre-BAA material, and wrote to the program manager for more information on the "counterfactual prediction" project. Awaiting his reply.

7.1.19 @11:12pm - DAGs are like optical lenses. First you use them as spectacles, to see things you already know, then you try them as telescopes and microscopes, to see things you never knew existed. Good luck my fellow econs; astronomy will never be the same. #Bookofwhy.

7.1.19 @12:22pm - (1/2) Mehdi Hasan's embarrassment will not end here. Once the conversation shifts to discuss the right of Jews to a homeland, he will be facing a dilemma: (1) To be honest and declare (like barghouti) that Jews are not a people or (2) To say that Jews, if well behaved, are entitled
7.1.19 @12:22pm - (2/2) to some semblance of sovereignty in the land. Theoretically, option (2) would get him off the hook but, unfortunately, he can't lose his support base of Arab rejectionists for whom such an admission amounts to a betrayal of 120 years of bloody wars and uncompromising denial.

7.1.19 @6:22am - The do-operator is not limited to physically "doing". It is an operation on your model of reality and it informs others about your model and about physical interventions that are feasible. On the interpretation of do(x), see: #Bookofwhy

7.1.19 @2:40am - (Replying to @LARichwine @NimaCNN @LAPressClub) Thanks @LARichwine for immortalizing this moment of grace. And thank you @NimaCNN for being part of a most memorable evening, and for honoring our son Daniel with your courage and integrity.

7.1.19 @2:32am - This quote, which sounds obvious in today's standards, was much debated in the 1980's, when AI folks labored to find proper formalisms to represent uncertainty in expert systems. This paper uses data insufficiency to expose a weakness in belief functions.

7.1.19 @1:48am - (1/2) We should all live to see Mehdi Hasan's face the first time he is called "Zionophobe" in front of an audience. On the one hand he is proud of being anti-Zionist, so he cannot deny the charges. On the other hand he will resist the analogy with Islamophobia which would force
7.1.19 @1:48am - (2/2) to philosophize on what "identity" is, be it religious, national or historical which, again, will steer the conversation to where we can win hands down: the moral imperatives of Zionism and the racist deformities of Zionophobia - the ugliest word in town.

7.1.19 @1:27am - (Replying to @EWilf @intelligence2) @EWilf , The title should have warned you of what the organizers tried to achieve: clearance from charges of AS. I stopped participating in such debates unless they change the title to: "Is Zionophobia racism?" or "Zionophobia on trial". ... "AS-- NO MORE"

7.1.19 @12:30am - (Replying to @zacharylipton) My goodness!! I never thought of becoming an icon of longevity, no way, playfulness yes, but longevity? Not in my worst dreams. Wait a minute! Perhaps playfulness is the secret to longevity? But isn't the latter simply a means to getting the former in greater quantity?#Bookofwhy

6.30.19 @2:58pm - (Replying to @IndexLlc @IARPAnews) Thanks for posting. I was not aware of this program and it is hard to tell from the BAA text whether its authors are aware of the fact that counterfactuals have been tamed, domesticated and algorithmitized, as in here . We need to find out. #Bookofwhy

6.30.19 @2:47pm - (Replying to @FJnyc @questionsin2014 @AshagerAraro) I share your suspicions. Zionophobic Jews-by-birth deny Jews rights that they grant to any other collective -- the right to define themselves. Does this make them racists? Painfully so!

6.30.19 @2:35pm - (Replying to @bnaibrithcanada) Thank you, @bnaibrithcanada for retweeting my speech to your followers. I am confident that, with your help, Zionophobia will become the Ugliest Word in Town. ps. Is there a Bnai Brith US? I wish they join us, in the trenches.

6.30.19 @2:34am - A transcript of a graduation speech I gave at UCLA last week is now posted on line: ... It explains what Jewish students are rallying for, and the kind of changes your campus will hopefully see next academic year.

6.29.19 @11:09pm - (Replying to @ShalitUri) Gee, I did not know about the Miao etal paper, perhaps because I was already immersed in #Bookofwhy. God bless Twitter for keeping us updated.

6.29.19 @3:39pm - (1/3) This paper identifies causal effects by proxies, a task shown feasible here and here . Blocking back doors is a sufficient condition for identification, not necessary; it turns out that, under certain circumstances, a proxy
6.29.19 @3:39pm - (2/3) can replace a blocker Z. Going from ATE to ITE ("individual" treatment effect) is not really hard (theoretically) if by "individual" we mean "c-specific effects" where c is a set of characteristics marking the individual, namely X=x. The nice thing about the paper you cited,
6.29.19 @3:39pm - (3/3) is that everything is spelled out in a language CI folks can understand, so that mysteries can be de-mystified. The authors should be commended. Effect-restoration was a breathtaking mystery to me in 2010, and it shows in the writing; I called it "far from obvious" #Bookofwhy

6.29.19 @12:42am - (1/2) To illuminate our discussion on PS, I am providing free access to Causality, Section 11.3.5 "Understanding Propensity Scores " . Note, in particular, Rubin's referring to PS matching "as if they had been randomized," and my closing remarks:
6.29.19 @12:42am - (2/2) "it is not enough to warn people against dangers they cannot recognize; to protect them from perilous adventures, we must also give them eyeglasses to spot the threats, and a meaningful language to reason about them." Readers of this Twitter understand it. #Bookofwhy

6.28.19 @12:54am - (Replying to @rlmcelreath) I've found that barriers among disciplines are lowered when we stress the questions to which CI seeks answers: (1) Effects of pending interventions, (2) Effects of undoing past events. PS. I couldn't read how the match-oxygen problem is related to attributable fraction #Bookofwhy

6.28.19 @12:16am - (Replying to @nghushe) Composure! Mom, Listen! This is the most complimentary message I received since my Bar-Mitzva. Three days after a prominent statistician accuses me of "calumny, caricature & confusion." I checked the dictionary, yes, the word exists; it just was not in my vocabulary. Thanks.

6.27.19 @11:21pm - (Replying to @edwardhkennedy) Fine. But I asked for a published paper that states so explicitly, to warn readers against assuming that PS has anything to do with asymptotic bias.

6.27.19 @11:17pm - (Replying to @jasonhartford @causalinf) HHMMM! you weren't talk about the real PRIMER ( ) ?? This too is a huge return on investment. For $20 and a few simple examples you get a glimpse at what causal inference can do for us, which is quite a lot. #Bookofwhy

6.27.19 @11:00pm - (1/2) I have no doubt that proving the asymptotic equivalence is immediate. Indeed, you can find such a proof in Causality ch 11 Eq. 11.10. What I asked however was whether anyone knows of a paper that actually states it EXPLICITLY, WITHOUT the assumption of non-confoundedness. The
6.27.19 @11:00pm - (2/2) paper you cite mentions Nonconfoundedness 17 times, thus contributing to the marketing myth that PS matching somehow contributes to bias reduction. See footnote 9, Causality p.349. #Bookofwhy

6.27.19 @5:14pm - I have not seen the connection, nor shown one. But I am a slow learner. The connection to complexity fascinated me in my youth: "On the Connection Between the Complexity and Credibility of Inferred Models," ... But I haven't tuned in since. #Bookofwhy

6.27.19 @3:19pm - Statistics as a discipline that helps us go from samples to distributions should be embraced and promoted. Statistics as an intellectual blinder that prevents one from seeing beyond distributions should be abandoned and shunned. #Bookofwhy

6.27.19 @7:11am - (Replying to @ildiazm) I never object to inclusion, especially inclusion of basic building block like stats. I object to exclusion. Like math teachers who exclude multiplication from arithmetic because they can always add a number to itself n times, i.e., the "classical way". #Bookofwhy

6.27.19 @2:46am - (Replying to @JDHaltigan) I agree with your take, and our challenge is to make 1>>2 in a climate where 2 control academia and demand submission from 1. #Bookofwhy

6.27.19 @2:39am - (Replying to @stephensenn) Exactly the way Lord describes what the 2nd statistician's does: Compare W_F of Diet-1 to that of Diet-2 for students of equal W_I. But Stephen, it is your turn now to teach me what I do not know about adjustment, I am listening carefully to new ideas, Just listening. #Bookofwhy

6.27.19 @2:21am - Thanks for posting, Jennie, now everyone can see what I mean by "causal inference" and why it needs a new logic, different from "the classical approach" we have been discussing here. #Bookofwhy

6.27.19 @12:41am - (1/3) I am curious, do you know of any paper that states explicitly the asymptotic equivalence of PS and the "adjustment formula"?? I have only seen it in Causality, ch 11, not elsewhere, Why? Perhaps someone is interested in marketing it as a magic wand? I am also curious to know
6.27.19 @12:41am - (2/3) how many readers hear this equivalence for the first time. It says that regardless of what covariates you use, as the number of samples increases, the bias of the PS estimator converges to the bias of the adjustment estimator. If many hear about it for the first time, it will
6.27.19 @12:41am - (3/3) serve as an example of information stifling that CI needs to liberate itself from. Anyone knows what the new CI textbooks say about PS? I would use it as a litmus test for authors understanding of modern CI. #causalinference #Bookofwhy @causalinf

6.26.19 @11:44pm - By all means. I hereby advocate, as always, that data-science rests on two equally important pillars: causal inference and statistical estimation. However, a glaring asymmetry can be seen today in academia: researchers in the CI pillars are thirsty for new
6.26.19 @11:44pm - (2/3) tools from the estimation pillar but not the other way. By and large, leaders of the stat pillars have zero interest in advances emerging from the CI pillar. The great majority of them truly believe in "do it the classical way" and "a causal model is a special case of a
6.26.19 @11:44pm - (3/3) predictive model". Moreover, in academia, the stat pillar dominates its CI partner by 100:1 ratio, and now insists on total dominion in the name of "its just a special case". Thus, to achieve equilibrium, I think CI needs academic autonomy, at least for a while. #Bookofwhy

6.26.19 @11:08pm - (1/2) Why the anger? Was I wrong in pointing to Gelman's blog as a "stronghold of statistical thinking"? (He has 21K followers, more than any other statistics-minded blog that I know.) Did I misquote it as advocating "do it the classical way"? or in stating that a "causal model is
6.26.19 @11:08pm - (2/2) a special case of a predictive model" or as advocating adjustment for all "pre-treatment differences among groups". I am always willing to learn more about what "adjusting" means. What is it? Teach us #Bookofwhy

6.26.19 @7:13pm - (Replying to @oacarah @PWGTennant) I think these papers miss the point. Agree? They treat PS as another method of identification, rather than an efficient estimator of the adjustment formula. #Bookofwhy

6.26.19 @7:01pm - (1/2) There was a time when statisticians were the guardians of prudence and caution, and causal folks the unruly children of unprincipled adventures. Today, if you peek at the strongholds of statistics (eg, Gelman's blog) you will find the opposite. CI folks hold the leash
6.26.19 @7:01pm - (2/2) on scientific principles, while statisticians advocate "do it the classical way", namely, rush into the mine-field of causation w/o a metal detector, and tell students that curve fitting is "causal inference". Yes, The Times They Are A-Changin' #Bookofwhy

6.26.19 @3:39pm - To summarize: (1) Is it just more explicit? No it is less explicit. (2) Helps with checking 'balance'? Sometimes, but 'balance' is the wrong criterion for covariates. (3) Encourages better model building? On the contrary. The advantage lies in mapping high-dim to 0--1 interval.

6.26.19 @3:39pm - In case you skipped Section 11.3.5 in Causality, titled "Understanding Propensity Score" (p.348), I would highly recommend it, for it clears up some of the myths connected with PS. This one may also be illuminating #Bookofwhy #EpiTwitter #Causalinference

6.26.19 @2:43pm - (Replying to @elizpingree @Jamie_Woodward_) Agree. Aside from its artistic qualities, the significance of the Lion-man in human development cannot be underestimated. Inspired by Harari's "Sapiens" , I featured it as the seed of counterfactual reasoning in the #Bookofwhy . Thanks for posting.

6.25.19 @11:57pm - (Replying to @lisabodnar) How powerful and truthful your uplifting words sing. I wish I could see them each time an academic troll unveils his/her pain. #Bookofwhy

6.25.19 @10:45pm - Thanks for your kind words. As to Aesthetics, our publisher insisted that general-purpose books should look different from technical books. Who are we to resist? #Bookofwhy

6.25.19 @10:40pm - (Replying to @adamdedwards) Good question. Thus far, I know of only two US philosophy dpts elevated to the age of causation: CMU and CalTech. Anyone knows of more? #Bookofwhy

6.25.19 @8:14pm - Agree, Epidemiologists are 98% there. All it takes is a final snip of the umbilical cord to mother-stat with a firm commitment to listen to what causal models tell us. #Bookofwhy

6.25.19 @1:13pm - (Replying to @DanielNevo) Same question can be asked about statistics. But it so happened that, in order to attract top scholars to the field, statistics insisted on academic autonomy, and dominion over data analysis, rather than being unappreciated minority in each data-using department #Bookofwhy

6.25.19 @3:06am - (Replying to @vkehayas) We have two additional sources of information: (1) playful manipulations (often called "interventions" or "experimentation") and (2) hearsay (often called "education"). #Bookofwhy

6.25.19 @3:01am - (Replying to @phi_nate) I twitted my take about two months ago, saying essentially: Can anyone translate what he is doing to our language so that we can prove that the claims do not violate any of the impossibility theorems we derived mathematically? Waiting. #Bookofwhy

6.25.19 @2:56am - (Replying to @zaffama) Agree, the culprit is in the interpretation. But notice an interesting phenomenon: the interpretation controversy did not rise when we accepted the axioms of probability, it has arisen only when we derived a consequence of those axioms, and interpreted it. What took us so long?

6.25.19 @1:50am - (Replying to @mariotelfig) We are talking about a one-line proof, not one-line statement of the theorem.

6.25.19 @1:48am - Eventually, I am sure, there will be more Causal Inference PhD programs than statistics PhD programs, possibly under the title "data science - causal inference" The question is which departments will launch it first, statistics or computer science?

6.25.19 @1:33am - Name another one-line theorem that has remained controversial for 250 years, and books like "the theorem that never dies" are written about it, and people are hired and fired in its name, etc etc. My take: it is not merely a theorem; it is a statement. #Bookofwhy

6.25.19 @12:26am - (1/2) Apropos Bayes. Does anyone thinks Bayes' Theorem is really a theorem? If it is, then it is the most trivial theorem in the cosmos, with a one-line proof. Can someone, even a Reverend, become immortal with a one-line proof? If it is more than just a "theorem", whence comes its
6.25.19 @12:26am - (2/2) added value? The #Bookofwhy answers this question (p. 102) from computational perspective, since I could not find any discussion of it in the statistical literature (@learnfromerror, @f2harrell, @stephensenn ) I believe it has more to do with psychology than with statistics.

6.24.19 @11:14pm - Agree. Bayesians first reacted to frequentists zeal, who persecuted them for contaminating statistics with "subjective knowledge". In time, they defined club membership by "priors on parameters", regardless if those priors conveyed knowledge or habits. I go Bayes 1763 #Bookofwhy

6.24.19 @8:58pm - (Replying to @kareem_carr @EpiEllie) Causal inference has been unified 10 years ago, see . True, the "schools" are still singing different anthems, laden with egos and tribalism, but you, as a rebellious champion of commonsense should look at the content, not the label. #Bookofwhy

6.24.19 @8:36pm - (1/2) My, My, Carlos, this paper was published 10 years ago and, so untypical of me, I am still behind every assertion. Remarkably, 10 years have passed, and statisticians are still resisting the distinction between causal and statistical notions (Section 2.1). Just this week,
6.24.19 @8:36pm - (2/2) Andrew Gelman @StatModeling wrote: "So I think it's a mistake to think of causal and predictive inference as being two different things." Your posting this review makes me both sad (0 inches - 10 years) and hopeful -- I detected sparks of awakening on Gelman's blog #Bookofwhy

6.24.19 @8:03pm - (Replying to @kareem_carr @EpiEllie) I wish you were right. My brief (83) encounter with statistics textbooks reveals a slightly different picture -- a complete prohibition on all assumptions, philosophical, causal or otherwise, except statistical assumptions that are but a tiny part of decision making. #Bookofwhy

6.24.19 @7:54pm - (Replying to @lisabodnar @EpiEllie) Congratulations, Lisa! Judea

6.24.19 @7:47pm - (Replying to @kareem_carr @EpiEllie) The tough question is: Is this a "well-defined" intervention?

6.24.19 @3:14pm - I think this discussion would benefit from a glimpse into the Bayesian vs anti_Bayesian controversy in AI, in the 1980's, about how to model epistemological uncertainties in expert systems. My recollections are here Biased, but honest #Bookofwhy

6.24.19 @4:28am - (1/2) When statisticians learned how to spray priors on parameters (of distribution functions) they formed an exclusive club called "Bayesian Statistics" and decided that he who won't spray priors on parameters is "not a Bayesian". My definition if Bayesianism is more broad, it
6.24.19 @4:28am - (2/n) follows Bayes paper of 1763, and it has to do with his interpretation of the phrase "given that we know X=x" and a license to invoke prior information. When I coined the name "Bayesian Network" (1995) I justified it on these grounds and I added Bayes' fascination with "cause"
6.24.19 @4:28am - (3/3) as another reason. Today I am only half Bayesian for reasons explained here , mainly because the bulk of our knowledge is causal, which cannot be captured by priors over parameters, definitely not parameters of a distribution of observations #Bookofwhy

6.24.19 @2:25am - True, I am very fond of my 1988 book "Probabilistic Reasoning", but I left Baeysian analysis in favor of causal inference, partly because most of medical reasoning is causal, not probabilistic. Its a whole new paradigm , and much fun #Bookofwhy.

6.24.19 @12:47am - (1/2) On a different occasion we will debate what DAGs cannot do, promise. Right now, the question is "Can a mechanism-seeking researcher answer causal questions from partial information alone, given in the form of "who listens to whom". Here are some examples:
6.24.19 @12:47am - (2/2) (1) The prisoner is dead. What if Rifleman-A refrained from shooting? (2) More people died from inoculation than from smallpox. Should we ban inoculation? (3) Is there a drug that is good for men, good for women, and bad for a typical person? #Bookofwhy

6.23.19 @11:35pm - (Replying to @yourbirlfriend @EpiEllie) @yourbirlfriend Fascinating! But cellular automata are driven by local forces and "minimum thread" is a global feature. How can you accomplish it? @yudapearl

6.23.19 @11:25pm - (Replying to @NeuroStats @shravanvasishth and 4 others) For computer scientists, the helplessness of Bayes analysis in causal reasoning comes glaring already in the notation. BDA invokes only one conditioning symbol, the vertical bar |X=x). Causal reasoning requires |do(X=x) Bingo! Done! You cant get "do" from "see" #Bookofwhy

6.23.19 @4:01am - (1/3) This article is worthy of our attention: ... 47 journal editors are offering guidelines to authors on ways to report results of causal inference studies. It is refreshing to see 47 editors reach consensus on a topic that only a decade ago was a sure ticket
6.23.19 @4:01am - (2/3) to discord. I believe the availability of DAGs as a communication language helped the process. I have trouble with some of the terminology (e.g., "causal association") but, overall, I welcome the timely rejection of "traditional" approaches of wholesale adjustment for
6.23.19 @4:01am - (3/3) everything one can conveniently measure. See @StatModeling for a lively discussion of opposing viewpoints, especially my explanation of why blindness to DAGs is an invitation to bias amplification . #Bookofwhy

6.23.19 @2:35am - (1/3) When you have a chance, please explain to readers on this Twitter how Bayesian Data Analysis (BDA) can help one think about causality. I have heard it from many statisticians and data analysts but I have never been able to understand what they find to be helpful and why.
6.23.19 @2:35am - (2/3) Is it the "model selection" part offered by BDA? Or the idea that you are properly combining prior knowledge with data? In my opinion, BDA is a siren song that lure people away from properly "thinking" about causation, as I argue here and in many
6.23.19 @2:35am - (3/3) other forums. I am appealing to you because, as an accomplished reader of #Bookofwhy I think you would be able to pin point to us where precisely BDA enthusiasts see the connection to Causal Inference, and why I am missing this connection. As always, a toy example is the KEY.

6.22.19 @7:59pm - (Replying to @RMartinChavez) I could not resist clicking on "like" to such a compliment, but the main thing is it is true: "At last, a science of causation!" And as much as I would try to minimize it the facts will scream at my face: a science of causation. #Bookofwhy

6.22.19 @5:58pm - (1/n) My remarks about being interested in seeing a Multi-level problem solved, and about traditionalists fearing toy-problems were in the context of the discussions on Gelman blog. As to #Bookofwhy, you are asking to be shown the "how" first and the debates second. This is exactly
6.22.19 @5:58pm - (2/n) what the book does. Chapter 1 tells you about the inference engine and what kind of questions it solves. What you see as a "fight" among competing approaches does not exist, because the history of debates about causation was not among "approaches" but among "ideologies"
6.22.19 @5:58pm - (3/n) 3/n that do not qualify for the title "approach". An "approach" should be armed with definition of what problems the approach attempts to solve and theoretical guarantees of reaching adequate solutions under certain conditions. Thus, the reason readers get the impression that
6.22.19 @5:58pm - (4/n) there is only one "approach", (ie SCM,) is that there is really only one approach armed with the needed theoretical guarantees, the alternative themes where merely "themes". Currently, I know of one competing "approach", ie, Rubin's potential outcome , which is logically
6.22.19 @5:58pm - (4/n) equivalent to SCM under the same assumptions but makes it hard for researcher to represent assumptions. It is described humbly and respectfully on pages 272-280. (I just made a similar point on Gelman's blog.) Its tough to tell the naked truth to readers who expect
6.22.19 @5:58pm - (5/n) something else. Some even view my claim about a transformative revolution to be nothing but "hype". I hope now, that you have finished the #Bookofwhy and had a chance to seek alternatives, you see that I had no choice but risk being called "hype" - "Si Muove" it really moves!`

6.22.19 @6:23am - (Replying to @djvanness) The calculus cannot help you with this choice, since you have not specified the state of knowledge upon which such choices are made. Now, since the calculus kicks in only after you made the choice, it can be considered an "assumption". #Bookofwhy

6.22.19 @5:42am - (Replying to @sakrejda @shravanvasishth and 2 others) You are being unfair by assuming apriori that what you see as "pettiness" is not genuine attempts to learn from each other. I, for one, would very much like to hear more about what we can learn from the "broader search" that goes beyond "causal inference". Care to explicate?

6.22.19 @5:31am - In causal modeling we do not call it a "choice" but an "assumption" (item (2)). If you think it is not plausible, go ahead and tell the calculus that Hall affects weight independently of Diet. (Perhaps one Hall has a dancing floor??). But we need a story to proceed #Bookofwhy

6.18.19 @5:35am - (Replying to @gjcampitelli @Lester_Domes and 4 others) I would stick to my SCM religion: Solutions must start by articulating the questions. What do we mean by "handling"? What are we expected to do after we differentiate between FE and RE in the model construction? Or, put more bluntly, why would anyone care? #Bookofwhy

6.22.19 @5:23am - It was great fun talking at your conference and assuming (wrongly) that audience never heard about causal inference. Thanks for posting this very slide "What is Causal Inference?" which is so timely in view of conversations we are having on Twitter and on @StatModeling #Bookofwhy

6.22.19 @5:09am - (Replying to @stephensenn) In Fig. 4 "Dining Hall " appears because it is taken from Wainer etal where it is another name for "Diet". Would you prefer we proceed with Fig. 4 and compute P(weight gain| do(Dining Hall)) ??? Give me a story. #Bookofwhy

6.22.19 @4:27am - (Replying to @stephensenn) Are you happy with the story of Fig. 6.9(b) ??. If not, please modify, if yes, we will go to: (3) data available. #Bookofwhy

6.22.19 @4:22am - (Replying to @stephensenn) These questions are vividly answered in Fig. 6.9 b. "Hall" is not in the graph, which means it is irrelevant to any research question. It shows W_I ----> D, which means "Diet may attract slim/heavy students", and so on, all assumptions are vividly displayed. #Bookofwhy

6.22.19 @4:04am - (Replying to @stephensenn) We need to analyze each version and see. So far we do not have a story -- what's the role of the "Hall". Is it just a place where Diet is served or a place that also affects weight or a place that attracts slim students, or a place that attracts students craving for a given diet?

6.22.19 @3:50am - (Replying to @stephensenn) Causal calculus does not "handle" things; it answers (1) research questions, given (2) assumptions and (3) data (experimental or observational). So far we got the question: (1) Find the causal effect of diet on weight gain, or P(gain|do(diet)). Now we need (2) and (3). #Bookofwhy

6.22.19 @2:23am - (1/2) I thought we agreed that in these versions Lord's paradox disappears. If you are still interested in "handling" these versions, independently of Lord, by all means, but you need to specify them as we specify any "toy problem", namely, (1) what do you wish to estimate?
6.22.19 @2:23am - (2/2) (2) What (causal) assumptions are you willing to make? and (3) what data is available to you? Let's start with ONE version, not TWO, to avoid going back and forth. #Bookofwhy

6.22.19 @2:01am - Unfortunately, I had to remove the Babylonian vs Greek analogy from the final version of the paper ; a reviewer insisted that this would offend ML folks. #Bookofwhy

6.22.19 @1:28am - Why do you assume that I am trying to "advocate' one approach or put down others? Why not assume that I am genuinely craving for ANY approach that enriches causal inference, and I can't satisfy this craving (my weakness) without seeing toy examples solved in other approaches.Why?

6.21.19 @10:48pm - I have the feeling that @StatModeling folks did try toy problems and the reason they dread them so consistently is that they realize they havn't got the tools. I would be very interested in seeing a simple Multilevel problem well posed and toy-like solved.

6.21.19 @9:55am - I like the term "susceptible to persuasion" (or "gullible"), because Ang Li has just finished a paper that formalizes this notion and gives it algorithmic teeth #bookofwhy

6.21.19 @12:37am - I just finished an adventurous visit to Gelman's blog ... and I am sharing here the last sentence: "if Causal Inference is 'Statistical Inference given causal assumptions' [as some claim] then Car making is car-painting given an engine and a body."#Bookofwhy

6.20.19 @10:30pm - Yes, Just in case you are at UCLA tomorrow, and have not gotten tired yet from listening to my Songs of the Revolution, join me tomorrow 3:30 pm where I will be singing one Aria to a slightly new libretto in front of social scientists and other mostly harmless folks. #Bookofwhy

6.20.19 @1:32am - I was truly honored to speak at the Algemeiner Gala, to be introduced by Sharon Stone and to warn against "Zionophobic Thuggery," which I hope will be recognized as the ugliest form of hostility on US campuses.

6.19.19 @4:52pm - @JewishPub. The more I read it, the more strongly I feel about including it in the list of 5 books that an "educated Jew" could not live without.

6.19.19 @3:49pm - (1/2) Many thanks. This is a good place to start chewing the eco. literature. One hurdle is the definition of "identified" which in eco. usually means identify the functions or their parameters, and in CI means identify a query Q which depends on the functions. I have just
6.19.19 @3:49pm - (2/2) noticed that the example you brought up is the usual IV setting, as in here , So it is already in our arsenal of "Powerful tools." Will see if more nuggets can be excavated. #Bookofwhy

6.19.19 @1:44am - (1/2) Great!! Here are some powerful results that hold for the linear extreme of the spectrum: . Can you or one of your students pin point to me which of them is extendable in some way to nonlinear systems? For example, Eq. (20) shows us how to estimate ANY
6.19.19 @1:44am - (2/2) counterfactual expression in ANY linear system. Can this capability be extended in some way to binary monotonic nonlinear systems? Just a hint, or eq.(#) would do, but please do not send me without guidance to a vast unchartered literature. Thanks #Bookofwhy

6.19.19 @4:30am - (Replying to @DanielOberski) There is a habit, especially on Twitter, by people who have not read things with sufficient depth to quote authors names, or provide links to cryptic papers and say: "It is treated here...". Done. "Without links" means: give me the basic idea, dont send me elsewhere. #Bookofwhy

6.19.19 @4:21am - (Replying to @WhitneyEpi @AdanZBecerra1 and 12 others) One should note that these ""classical" rules from the RCT context were merely "rules of thumb", lacking theoretical underpinning. It turns out that some post-intervention factors are actually safe, while some pre-intervention factors are unsafe. See Causality p. 339 #Bookofwhy

6.19.19 @3:14am - (1/3) I am partially familiar with Matzkins works, and I know that many economists refer to them. However, I have not been able to supplement the arsenal of powerful results now available for linear and NP extremes of the spectrum, with similar results applicable to mid-spectrum
6.19.19 @3:14am - (2/3) I examplified what I mean by "powerful results" here: ... Perhaps you or some other expert economist can fill me in on what is known about the mid-spectrum. Or, what is the one most important identification opportunity one would miss by not
6.19.19 @3:14am - (3/3) this literature as thoroughly as it deserves. For example, is the counterfactual estimability of linear models extendable in some way to monotonic models? or Binary models? #Bookofwhy

6.18.19 @10:43pm - I have posted a comment on Gelman's blog, explaining why causal and predictive inference are not the same thing, and why it is beneficial to solve each task in its own distinct vocabulary. #Bookofwhy

6.18.19 @3:03pm - (1/2) "Powerful results" are those that one remembers. For example, "Every causal effect in every NPSEM is either identified or quickly proven to be non-identifiable". "Every linear SEM that contains no bow-arc is identifiable" "Every counterfactual in linear SEM is estimable" etc.
6.18.19 @3:03pm - (2/2) Are there similar results for the spectrum between linear and nonparametric? For example, what do we know about the loglinear variety? What identification results from linear SEM are preserved? To be "powerful" they need to be articulable verbally w/o links . #Bookofwhy

6.18.19 @12:30pm - My thoughts? Hilarious!!! Here is one gem: "So I think it's a mistake to think of causal and predictive inference as being two different things." Here is mine: "So I think it's a mistake to think of any two things as being two different things - it's all arithmetic" #Bookofwhy

6.18.19 @5:35am - (Replying to @gjcampitelli @Lester_Domes and 4 others) I would stick to my SCM religion: Solutions must start by articulating the questions. What do we mean by "handling"? What are we expected to do after we differentiate between FE and RE in the model construction? Or, put more bluntly, why would anyone care? #Bookofwhy

6.18.19 @4:49am - (Replying to @fr_amodeo) If by "handling" you mean "provide distinct specification" then for the standard SCM the answer is NO. But why not add a marker "L" to any family that combines linearly, and leave the others unmarked. The beauty would be to decide if an arbitrary causal effect is fixed #Bookofwhy

6.18.19 @2:12am - (Replying to @juli_schuess @PHuenermund) Paul, why the wonder? This is the classical IV setting and, as @juli_schuess notes, all parameters are identifiable using either Wright rules or the method of moments. For generalizations, I would use #Bookofwhy

6.17.19 @8:50pm - (1/2) Got it. Thanks. And I am retweeing because many CI folks are likely to stumble on this jargon. Writing y=bx+u gives fix-effect, because P(y|do(x+1))-P(y|do(x)) is independent of u. But writing y=f(x,u) gives random-effect. So, nonparametric SCM assumes random effects, and the
6.17.19 @8:50pm - (2/2) distinction does not show in the DAG, only in one's declaration "Assume a linear model". An entirely different question is: "Is there a test that detects the presence of RE in a given population?" Yes, there is, and beautiful one too: #Bookofwhy

6.17.19 @7:59pm - (Replying to @Corey_Yanofsky @Lester_Domes and 2 others) A partial meeting of minds was achieved here, on Twitter, when my mind realized that what Senn objects to is the perfect alignment of the two ellipses, which would rarely occur in real life. This I consider to be orthogonal to Lord's question: "Who is right?" #Bookofwhy

6.17.19 @7:52pm - (Replying to @Lester_Domes @Corey_Yanofsky and 2 others) I will try, if only I knew what RE and FE are. (With examples please). Are the the causal effects estimated in an ideal RCT RE or FE ????

6.17.19 @6:59pm - (Replying to @PWGTennant @lisabodnar @ProfMattFox) Sorry to be missing your dazzling party. Now I have two reasons to attend SER-2020. Cheers!

6.17.19 @1:15pm - So, according to your perspective, modern SEM maintains the same input-output relation as SCM. Namely, INPUT = data + qualitative causal assumptions. OUTPUT= causal effect sizes. And no one is uncomfortable about the word "causal" there, even in published papers. #Bookofwhy

6.17.19 @12:59pm - (Replying to @andres_fandino) There is no such notion as "appropriate". If your understanding of the problem demands N exogeneous and M endogeneous variables, so be it. Reality comes first, estimability second. #Bookofwhy

6.17.19 @12:43pm - (1/2) Yours is a very encouraging perspective of modern SEM, namely, SEM = "SCM that accommodates parametric assumptions." Its badly needed. I am familiar with the linear and nonparametric ends of the spectrum; are there any powerful results in between? Say binary, or monotonic?.
6.17.19 @12:43pm - (2/2) Also, just out of curiosity, what percentage of your colleague/students can answer "Tell me if the partial correlation R_{XY.Z} is zero", or "Tell me which parameter is estimable by OLS". I have not communicated with them for several years. #Bookofwhy

6.17.19 @4:50am - (Replying to @y2silence) Beautifully put. And I wish more people learn to appreciate this miracle, which is truly unique. In fact, what else can one ask of a modeling methodology? #Bookofwhy

6.17.19 @3:28am - (1/3) In the last conversation I had with Peter Bentler he defined SEM as a "compact and meaningful representation of a covariance matrix". We know that "meaningful" is a round-about way of saying "causal". Fine. But since the purpose is declared as fitting a covariance matrix,
6.17.19 @3:28am - (2/3) one should ask: To what use can one put the conclusions of an SEM study (aside from getting your dissertation published in Journal of SEM), which usually comprises a huge list of path coefficients with their confidence intervals. I wish someone can explain why not list the
6.17.19 @3:28am - (3/3) estimated covariance coefficients themselves; a much simpler task. #Bookofwhy

6.16.19 @11:58pm - It is hard to discuss SEM since its practitioners are still not sure what it is. I tried to describe their confusion here and sway them to use SCM. Not successfully, and Psychomentika-type papers are my witnesses. Interesting social phenomenon #Bookofwhy

6.16.19 @11:29pm - (Replying to @ulusdd) True, but I dont know anyone who starts with a cyclic system and expects DAGs to handle it. DAG techniques are expected to compute properties of the input model, namely directed-acyclic systems. Besides, d-separatioin holds in linear cyclic systems. #Bookofwhy

6.16.19 @11:22pm - My perspective today: SEM is a community of researchers using SCM who refuse to commit to the causal reading of their models, argue endlessly about what that reading is, and refuse to benefit from the comp. power of DAGs. Too harsh? Would welcome opposing evidence #Bookofwhy

6.16.19 @11:09pm - (Replying to @omaclaren) I am not familiar with "rate parameters" "ODE models" and 'nonlinear least square". Can you explicate in terms of the input model, which is a DAG, be it linear or nonparametric. Of course a DAG cannot compute how much money you have in bank account, or halting problems etc.

6.16.19 @8:57pm - (1/2) "tools with which to think about models" is only one usage of DAGs. Another usage is computational; DAGs permit us to answer questions which otherwise are intractable. E.g.,"Tell me if the partial correlation R_{XY.Z} is zero", or "Tell me which parameter is estimable by OLS"
6.16.19 @8:57pm - (2/2) I am surprised that the computational capacity of DAG is under-appreciated by most researchers. Am I exaggerating? Is there another computational tool with which the questions above can be answered? Is there a question you wish to ask which DAGs do NOT answer? #Bookofway?

6.15.19 @6:28pm - (Replying to @robanhk) I love those "intellectuals" and have written a poem in their honor

6.15.19 @6:26pm - (Replying to @robanhk) I was not sure if you were sarcastic or honest. Now that you are equating "not loyal to Israel" with "denying Jews the right to a homeland" you made it clear. I do not know anyone who denies a people's right to a homeland who is also considered an organic part of that people.

6.15.19 @4:21pm - (Replying to @robanhk) Yes. I thought that the fringe group of Jewish-born intellectuals who deny Jews the right to a homeland have not felt targeted by Abdulhadi's rants; they run for safety by showing loyalty to her Zionophobic movement and do not need my encouragement.

6.15.19 @2:12pm - Excited towards keynote addressing the UCLA Jewish Graduation Ceremony on Sunday. Hoping to inspire the very students who were recently labeled "White supremacists" by a super BDS ideologist (still at large, still not condemned).

6.15.19 @1:28pm - (1/2) If instead of criticizing the way SEM's ARE used we take the trouble to explicate how they SHOULD be used, we will, I believe, end up with SCM's (Structural Causal Models) like those described in #Bookofwhy or . But,
6.15.19 @1:28pm - (2/2) now that we are discussing classical SEM, I am curious to know whether there is still a large community of SEM users who have not switched to SCM ??? If there is, can someone act as its spokesperson and explain what, aside from habit, prevents them from making the transition?

6.15.19 @2:35am - Sewall Wright had much harder time communicating with statisticians. He did not have the RCT metaphor to open the first window to their hearts, and he did not have the logic of counterfactuals to defend the structural assumptions behind his diagrams. A true hero. #Bookofwhy

6.14.19 @3:39pm - This is truly a gem -- thanks for posting. I have not read it for many years. And now that I re-read it, it covers so many topics that we have discussed here on Twitter. It even discusses modern hangups with "mimic RCT or else your causal effect is "not well defined". #Bookofwhy

6.14.19 @11:44am - The correct link to "Causal Foundations of SEM" is this There was a missing space. #Bookofwhy

6.14.19 @3:43am - Regarding the causal foundations of SEM, I have found another article that addresses this issue head on: . It also explains why teachers of SEM are so helplessly confused about a problem so simple. #Bookofwhy

6.14.19 @2:12am - Thank you, Rabbi Dunner, for immortalizing our meeting at the Algemeiner Gala. I felt truly fortunate to be able to contribute to our three common ideals "Truth, humanity and Jewish peoplehood" - a spec of sanity in the age of BDS madness.

6.14.19 @1:37am - Many have had the same SEM experience. The answer is that no magic dust is needed; the coefficients were always causal, and the association just help us to quantify those causal relationships. Bollen and I discuss it in "Eight Myths of SEM" #Bookofwhy

6.11.19 @8:15pm - Thanks, Lisa, for letting us know that you will MC tomorrow's Gala. Looking forward to seeing you again. And may the angels bless you and the other warriors in the trenches.

6.11.19 @6:25pm - @causalinf Have you looked into the way Peter Steiner represents RDD in DAGs? I have a link to his 2017 paper: ... #Bookofwhy @PHuenermund , @eliasbareinboim , @yudapearl , @Jabaluck @EpiEllie , @paulgp

6.11.19 @6:11am - (Replying to @TuomasPernu) I feel the same way, except that I found solace in the metaphysical assumption that reality acts as a "society of listening agents". So far, I have not found a more satisfactory metaphysical theory in the philosophical literature. Would be curious to examine alternatives.

6.10.19 @3:49pm - (1/2) "difference-making" or "counterfactual dependence" are various names philosophers used, but they fell short of operationalizing this relation through the simple mathematical concept called: "a function". I dont know what book you have on shelf, but if it is #Bookofwhy or
6.10.19 @3:49pm - (2/2) or or Causality, then you will satisfy you quest to see coherence among the assumptions and the conclusions. The philosophical literature, unfortunately, is still hung up on "what do we REALLY, REALLY... mean by CAUSE". Hopeless

6.10.19 @2:40pm - (1/2) One gift modern causal inference has given to us is a clear answer to your question: "what the assumptions and stages are, and how they relate to each other." See eg #Bookofwhy. Another gift is seeing all those assumptions expressed in terms of one primitive relation:
6.10.19 @2:40pm - (2/2) "One variable listening to others". One may argue that this primitive is just a causal relation in disguise. Perhaps, but it is still a gift to see ALL assumptions and ALL conclusions emerging from ONE easily grasped idea. #Bookofwhy

6.10.19 @1:55pm - Not entirely unfair. It lends support to the idea that perhaps "listens to" is an irreducible primitive in our mind which will stay irreducible until we can map the neural paths that are activated when we ask "why".

6.10.19 @1:53pm - (Replying to @PHuenermund) Not entirely unfair. It lends support to the idea that perhaps "listens to" is an irreducible primitive in our mind which will stay irreducible until we can map the neural paths that are activated when we ask "why".

6.10.19 @5:21am - (Replying to @fpgil) And I understand the Portugal is celebrating today its independence day -- the oldest independence day in Europe. Monday, 10 June: Dia de Portugal. I just heard it on Israeli TV, so, from Israel with love: Two small countries with rich rich history.

6.10.19 @4:09am - (1/2) I an not sure whether Woodward is satisfied with the structural account, see , but the philosophers that I read do (eg Spohn, hitchcock,Glymour). What is the point of asking for "the nature" of causation withouot telling us what type of answer will be
6.10.19 @4:09am - (2/2) accepted? The structural account, based merely on the notion of "listening to" is as satisfactory to me as any that I have seen in philosophy. And, in addition to its metaphysical satisfaction it can also solve the Simpson's puzzle, how can you ask for more? #Bookofwhy

6.10.19 @3:30am - Portuguese!! My goodness!! Now I can start arguing with all my Brazilian students on equal footing. Thanks for posting. #Bookofwhy

6.10.19 @3:21am - (1/2) The #Bookofwhy solution makes NO ASSUMPTION beyond what is specified by Lord himself. If you prefer to fantasize a complex multi-hall version of the problem, and fail to collapse it into Lord's distribution, do not blame Lord, nor #Bookofwhy . The key point is that, for
6.10.19 @3:21am - (2/2) the simple two-hall problem, each serving ONE diet, the solution provided in #Bookofwhy p. 217 is valid, and resolves a paradox that still baffles many good minds, even today, including many statisticians. Again, no extra assumption beyond those given by Lord and Fig. 6.9b.

6.10.19 @2:53am - (Replying to @stephensenn @analisereal) The distribution is given to us by the two ellipses; This is Lord's construction, not ours. NOTHING ELSE is assumed about variances and covariances. Mixed models were invented to help in the construction of distributions, not for handling a fully specified distribution.#Bookofwhy

6.10.19 @12:14am - The less background the better. Except, if you have not taken any stat-101 or econ-101 classes you will miss the fun of asking: "How come my professor never told us that causal inference is easy?" #Bookofwhy

6.10.19 @12:02am - (1/3) The #Bookofwhy is not about "what causal calculus cannot do" (eg, play chess, translate languages etc) but about the many miracles it CAN do. Among them resolving the simple version of Lord Paradox, with two dining halls, each serving one diet, and a very large sample. So,
6.10.19 @12:02am - (2/ ) believe it or not, but this simple version is still paradoxical to most mortals, and has been paradoxical for half a century. It is now resolved by causal calculus. Your multi-hall version may be of interest in a certain context, but I cant understand why you are insisting
6.10.19 @12:02am - (3/3) that this idiosyncratic version is essential for resolving the simple version, and that he who does not attend to your version is guilty of neglecting the foundations of statistics or worse. I do not buy it. Lets focus on the simple version -- are you happy with the solution?

6.9.19 @11:16pm - Good news for fans of #Bookofwhy. The book is now available in paperback ($14) from amazon: and, more importantly, friends tell me they got it delivered next day and it is much easier to read, handle and carry on the train. Tell your grandchildren.

6.9.19 @8:22pm - (1/2) I find VanderWeele's decomposition somewhat artificial. The two clinically meaningful components of mediation are: (1) The extent to which observed effects would be PREVENTED by disabling the mediating path and (2) The extend to which observed effects would be SUSTAINED
6.9.19 @8:22pm - (2/2) with the direct effect disabled. These two components (necessary vs. sufficient) collapse in linear systems, but are distinct when interactions are present. Each can be estimated using the mediation formula, , and , #Bookofwhy

6.9.19 @5:19pm - Congratulations, again, and I am re-tweeting your vision statement here, because it coincides so perfectly with my conception of where causality-land is heading. There may be some hidden cause in action here, but reality prevails regardless of models. #Bookofwhy

6.9.19 @5:00pm - (1/2) (Replying to @jwbelmon @jon_y_huang and 3 others) Some people use an "arrow on arrow" notation to indicate effect modification, but I find it unnecessary and confusion, because, if A modifies the effect of B on C then, from basic logic, B must modify the effect of A on C. In linear systems we can just use a product term
6.9.19 @5:04pm - (2/2) (Replying to @yudapearl @jwbelmon and 4 others) and in non parametric system, every parent is by default assumed to modify the effect of all other parents. I you need to find the degree of effect modification, you can use counterfactual logic, as VanderWheele is doing, or as we do in causal mediation #Bookofwhy

6.9.19 @3:26pm - (Replying to @stephensenn @analisereal) "The sample size is only two" My goodness!! we must be thinking about a different universe. I do not deny the existence of your universe but, given that I naively assumed that the two ellipses show a very large sample size, please describe your universe slowly slowly #Bookofwhy

6.9.19 @2:37pm - (1/n) Now that we a research question, we go to the next step: What data we have available. The answer is given by the two ellipse, showing (observational) measurements taken on 3 variables W_I W_F and D (Diet), no separate data on Hall. It is quite conceivable that each Hall #Bookofwhy
6.9.19 @2:37pm - (2/2) serves several diets, but the data does not provide separate measurement on H vs. D, so we assume Hall determines Diet unambiguously. Can we continue from this assumption? Or you prefer to introduce distinction between H and D.

6.9.19 @2:24pm - I second it! It is a great summary of the field, and makes the debate about "in statistics" or "extra-statistics" hollow and irrelevant. #Bookofwhy

6.9.19 @7:46am - (Replying to @stephensenn) Ring around the Rosie .... and we still do not what your research question is. A hint, perhaps? or a clue?

6.9.19 @7:20am - Perhaps Fisher/Nelder/Bailey/Speed/@PhilDawid can articulate what your research questioin is? I would gladly give them full credit for the ideas of symmetry/exchangeability if they can use them to solve your problem w/o knowing what you aim to estimate. #Bookofwhy

6.9.19 @7:09am - I cannot understand the issue without knowing three essential elements: (1) What you wish to estimate, (2) what assumptions (if any) you make and (3) what data you have available. Once specified, all "issues" are resolved mathematically. #Bookofwhy

6.9.19 @6:23am - (1/2) DV Lindley was a devout Bayesian, but he was also the first to understood that if (in Simpson's paradox) the same data leads to two different conclusions depending on the story, then Bayes analysis is helpless, because Bayesian methods are propelled by fitness to data. I have
6.9.19 @6:23am - (2/2) written about this in "Why I am only a Half-Bayesian" . It's a good paper because many Bayesians harbor the illusion that, if you only spray priors on models and take sufficient data, the posterior will peak around the correct model. #Bookofwhy

6.9.19 @4:54am - (Replying to @stephensenn) If you aim to analyze experimental studies from two different studies, on two different group, lets do it correctly, along the theory of data fusion. eg . Again, we need to articulate question, assumptions and type of data available. Ready? #Bookofwhy

6.9.19 @2:26am - (1/2) I am saying that the causal story behind the data determines which statistician is correct, and that the story should be articulated in the language of diagrams (not as Rubin and Holland told it). Thus, if we believe that diagram 3 is what Wainer etal meant, we go to diag 3.
6.9.19 @2:27am - (2/2) if we believe that Diet is different from Hall, and that another story describes how data were generated, we can draw the corresponding diagram and interrogate it regarding the correct analysis. Lets do it. Which diagram do you like? What is your research question? #Bookofwhy

6.9.19 @1:31am - I do not understand expressions such as "be all and end all"; I have not used any, nor implied any. As to your question, the diagram on p217 already contains "Hall"; it coincides with "Diet", exactly as presented in Wainer etal. Nothing changed. Just causal modeling. #Bookofwhy

6.9.19 @1:01am - (1/2) It is no longer secret. The lecture that @eliasbareinboim gave at Columbia last month ... was actually a job-talk and I wish to congratulate Elias on accepting a faculty position at Columbia, starting July 2019. I wish also to
6.9.19 @1:01am - (2/2) congratulate my colleagues at Columbia for strengthening their ranks with a top innovator of causal-inference research. May this marriage lead to major breakthroughs and smarter machines. Amen! #Bookofwhy

6.9.19 @12:26am - (1/4) I believe this set of slides reinforces what I tweeted earlier: ... -- its hard to cut the embilical chord to Mother-Stat. It occurred to me that this urgency to stay in Stat-womb was also the motivation behind the potential-outcome framework. The benefits
6.9.19 @12:26am - (2/4) were obvious, nothing is new, Y_1 and Y_0 are ordinary variables, with some missing values, so what? Everything else is ordinary statistics. The price, of course, was (1) Everything was tied to experimental "treatments", not to "events" or absence of events, and (2) we need
6.9.19 @12:26am - (3/4) to express knowledge in the language of {Y_1, Y_0} , namely, in the formidable language of "conditional ignorability". Some would argue: What's wrong in letting statisticians broaden the scope of statistics and then believe that "it is all statistics"? I believe PO is a good
6.9.19 @12:26am - (4/4) of what could go wrong. Distinctions play a role in science. Treating the Ladder of Causation as one chunk, in the name of "It is all statistics", while ignoring theoretical barriers between the 3 levels creates more confusion than the stat-womb warmth can sooth. #Bookofwhy

6.8.19 @9:08pm - (1/1) You seems to oscillate between your desire to validate the model and your desire to answer some causal question, eg. "is there a non-zero effect of Z on Y". Which of the two you wish to start? As @EpiEllie explained, conditioning on the mediator may lower or amplify
6.8.19 @9:08pm - (2/2) the association between X and Y, so this test is out. (Using your notation) If you can put down two competing models, one can tell you right away if there is a test that can distinguish them apart. The name of the game is to confess and let the mathematics work #Bookofwhy

6.8.19 @8:15pm - (1/2) Whether a problem area is causal or not depends not on the diagrams we can draw or intervention we can perform. It depends only on the research question we aim to answer. In my l last encounter with fMRI researchers I was disappointed by their lack of understanding of causal
6.8.19 @8:15pm - (2/2) inference and blind allegiance to potential outcome vocabulary. See . Are these authors representative of the field? If not, please describe what Dynamic Causal Models in fMRI are aiming to estimate, and one simple example of such a model #Bookofwhy

6.8.19 @7:57pm - (Replying to @jwbelmon @sjalexander @EpiEllie) @jwbelmon. I will try to help, but I do not understand what you want to do. If you can specify: your research question + your scientific assumptions, then whether it can be done or not can be answered mathematically regardless of what the field has been doing. #Bookofwhy

6.8.19 @7:49pm - @EpiEllie has done a good job describing the assumptions behind IV methods and their wide applications. I commented recently ... that epidemiologists are now proficient in describing IV methods in DAG's language. The next step is to REPAIR bad IV's #Bookofwhy

6.8.19 @2:50pm - (Replying to @y2silence @SALubanski) Most importantly, do not miss Carlos's final comment, which puts things in the proper perspective.

6.8.19 @5:29am - (1/n) (Replying to @jd_wilko) It is not really "provocative", just a gentle way of luring statisticians to cut the umbilical chord from Mother Stat. Instead of surgical do-operator, you condition on a variable F (force) which does the surgery for you. I used it in 1993 when
6.8.19 @5:43am - (2/n) (Replying to @yudapearl @jd_wilko) I thought that statistician are not prepared for a surgery. Other researchers, too, labor to create the illusion of remaining in the stat-womb. E.g., Heckman etal created a fix-operator to enforce this illusion . The folks at @harvardEpi still teach CI
6.8.19 @5:56am - (3/3) (Replying to @yudapearl @jd_wilko @HarvardEpi) 3 by "imitating RCT's". It is a tough umbilical chord to cut. Denis Lindley was the only statistician I met who said: CUT! In all these schemes we still need to import information from outside the data, which is the key to realizing that we are out of the stat-womb. #Bookofwhy

6.8.19 @4:18pm - Good luck and Happy landing! I am not exactly sure what @AVORA does or how, but when I see people taking causal modeling seriously, I know that #Datascience has a future beyond curve fitting. I will be looking forward to seeing example applications. #Bookofwhy

6.8.19 @3:51am - (Replying to @dlmillimet @PHuenermund) Great blog to teach us how "real world econonomists" think and work. Perhaps it is a good way of slowly introducing econ. readers to advanced methods, using graphs. E.g, the "IV with Endogenous Control" is solved here Others in

6.8.19 @2:49am - (Replying to @mimblewabe @viktorklang) Doubly agree. And this calls for a theory of explainability, namely, what is it about the CONTEXT of the conversation that makes us interpret "why" one way or another, say, "what for" vs. "by what means" #Bookofwhy

6.7.19 @7:09am - The closest would be the talk I gave at USC a few months ago: ... Some poetry, but mainly somber causal wisdom. #Bookofwhy @thinkmariya

6.6.19 @11:46pm - @_MiguelHernan A different take on Cochran's contributions to #observational studies is available here: , which emphasizes his "explanation of the mechanism by which the effect is produced" rather than "imitating" RCT's, which #Bookofwhy advises against.

6.6.19 @9:40pm - (Replying to @rakotesh) The flyer says: CIO UCLA Health Sciences IT TOWN HALL
Friday, June 7, 2019 10:00 AM to 12:00 PM
Pauley Pavilion, 301 Westwood Plaza
Arena Floor / East Bleachers
Enter through the NORTH EAST doors.
I will ask them about streaming, but you may have more clout.

6.6.19 @7:11pm - @EPIDEMIOLOGYY @mendel_random Now that epidemiologists are proficient using DAGs to describe selection bias: ... they will be happy to know that methods of repairing such bias are also available at the nearest DAG's brewery: , #Bookofwhy

6.6.19 @2:33pm - (Replying to @Abraham_RMI @_MiguelHernan @davidlederer) Not reporting would have been an option a few decades ago. Not these days, when we can explicate the assumptions behind our ES estimates and submit them as part of the report: If you buy assumptions XYZ, you must buy conclusions ABC. Conditional conclusions have value #Bookofwhy

6.6.19 @2:13am - If you happen to be at UCLA on Friday, 11 am, you are invited to attend a talk of mine in which I will try to summarize #Bookofwhy to health scientists, in 20 minutes of slides and poetry. See you there.

6.6.19 @1:33am - (1/2) Glad you plan to include precise conditions in your paper and, hopefully, relate them to identification theory. For example, gives examples of when surrogate experiments can decide Q and when they can't. It would be nice to simulate your method
6.6.19 @1:33am - (2/2) on such examples and see if the existence of h coincides with identifiability of Q (using the corresponding interventional data). Another interesting comparison would be to see how your h works when Q is identified from the (set of) graphs inferrable from the data #Bookofwhy

6.5.19 @11:36pam - (Replying to @mekarim7) Thanks, this was fixed already in our errata sheet.

6.5.19 @11:07pam - Suppose our target quantity is Q=P(y|do(x)), how would we get it from h(x)? Moreover, suppose our datasets are P_i(y|x1, for many many i's, x's and z's, Is Q necessarily identifiable from the available datasets? What happens if it is not? #Bookofwhy

6.5.19 @6:01pm - (Replying to @wgeary) The most inhumane atrocities in the history of mankind were committed by those who saw humanity on one side, and one side only. Now watch BDS puppets chanting: "human rights", "human rights", they can's see humans on the other side. Puppets can't see. They can only chant.

6.5.19 @4:15am - You are not alone, Petros. It is really the best introduction to causal inference that I know. And even the fear of sounding self-promoting will not deter me from saying so. One must be honest when so many communities are still under the boot of traditional education. #Bookofwhy

6.5.19 @2:04am - (1/2) Colleagues keep asking for my opinion on Bottou's method of revealing how the world works: ... I refer them to my May 12 Tweet, where I pleaded with followers to explain how he extracts invariances using training, but to explain it in the language
6.5.19 @2:04am - (2/2) of conditional, or interventional probabilities, P(y|x,do(z)...) , because, all we can get from training are such probabilities. So far, I have not received an explanation, so I remain speechless and mighty curious. Does anyone understand it? #Bookofwhy

6.5.19 @12:45am - @jjz1600 Hi James, Excuse my innocence, but what exactly is wrong with the NYT AD? I was there, in 1948, and I can verify every sentence in the AD as factual and well documented. I would welcome the opportunity to tell you personally what Tlaib's ancestors did to mine. @DrMikeH49

6.4.19 @9:02pm - (1/2) My puzzle is: How many of those econ seminars are aware of the fact that, in addition to "touching" on issues, a large chunk of them can actually be "solved"? I am not asking this question to prove anyone wrong. I REALLY DO NOT KNOW. It is already a year after #Bookofwhy, so
6.4.19 @9:02pm - (2/2) I am sure some econ. students are doing more than "touching", perhaps even with encouragement of their professors. But I really do not know the extent of this enlightenment. I don't see it in the universities that I watch, and I have no way of judging, unless you help me.

6.4.19 @4:34am - I got a review of Ian Stewarts forthcoming book Of chaos, storms and forking paths: How does statistics help us to understand the world? by Andrew Gelman ... My natural reaction: statistics and "understanding the world" are two different things #Bookofwhy

6.4.19 @3:15am - @jo_mendelson Glad you posted this quote from ... It is so easy for university administrators to hide behind "academic freedom" and theological definitions of anti-Semitism, and forget that thousands of creative academics have been criminalized on their campus

6.4.19 @2:50am - Remember Ang Li paper on how to select customers (or patients, or voters) that are most likely to respond to your request/action, though "responsiveness" is an unobserved, counterfactual notion? A revised and improved version is now posted here: #Bookofwhy

6.4.19 @12:27am - (Replying to @yudapearl @dbweissman) 2/2 namely, in econometric, but are afraid to do so even when needed? Or, to put it more mildly, what economics department is likely to prepare students who KNOW how to solve such problems? Does #Bookofwhy underestimate what these students actually know? Genuinely curious.

6.4.19 @12:14am - (Replying to @dbweissman) 1/2 OK, I am willing to reconcile with the existence of the gap between what people know and what they must do by their "fields". Fine. Can we conclude then that in the top economics departments in the US, students DO KNOW how to solve elementary problems in causal analysis

6.3.19 @5:00pm - (1/3) #Bookofwhy from the viewpoint of an enlightened economist. As you can probably guess, I am particularly interested in your comment: "His [Pearl] grasp of what economists, for example, understand ..
6.3.19 @5:00pm - (2/3) and don't understand about causal relationships is incomplete,". What is it that economists DO understand and that I assumed they DON'T? Has this part of "econ. understanding" been expressed formally in the econ. literature since Haavelmo and Cowell's Com.? Can economists
6.3.19 @5:00pm - (3/3) solve the toy problems posed to them here: ? I am genuinely trying to understand what they know that they are laboring to hide from us. E.g., Do they know which parameters can be identified by OLS? Which models have testable implications? etc. etc,...

6.3.19 @12:51am - (Replying to @BD_Zumbo @AlinaVdav) I salute your courage in joining commonsense, it is a dangerous road, as you must have felt already, but your students will thank you forever for saving them from outdatedness. #Bookofwhy

6.3.19 @12:30am - (Replying to @chandra1250) I really do not know, I am just a scribe. Perhaps the publisher knows.

6.3.19 @12:28am - (Replying to @KumailWasif) Thanks for you kind words. I will keep writing, but I need your help in expanding its potentials beyond my limited horizon. Thanks #Bookofwhy

6.3.19 @12:22am - (Replying to @NandoDF @dlowd and 5 others) I have not read Pinker's new book, but I thoroughly enjoyed his "Enlightenment Now". Perhaps because I am a born optimist, or because it brought to my attention data and insight I was not previously exposed to. #Bookofwhy

6.3.19 @12:17am - Your recommender must have been an extremely keen book-reviewer. I happened to have read this book already and, strangely, I enjoy reading it again and again. No dog in the fight, just the pleasure of a child listening to Aesop's fables for the tenth time. #Bookofwhy

6.3.19 @12:00am - (Replying to @riazshahzain1) Thanks for writing. And may that it inspires you to write an even better one.

6.2.19 @11:57pm - If I have encouraged even one professor/student at NYU to voice his/her knowledge and convictions about Israel, or to speak against the way Zionism is maligned on some BDS-occupied campuses, I would consider this Award the greatest honor of my life. Thanks!

6.2.19 @1:02pm - (Replying to @RevDocGabriel) Yielding to friends' wisdom. Will move to "incomplete". Thanks

6.2.19 @2:09am - (Replying to @DanielWhibley @EpiEllie @HarvardChanSPH) Note the difference. In #Bookofwhy you think: "who listens to whom?" In @HarvardChanSPH you think: "What if I intervene?". Luckily the difference does not affect practice until you get a variable like "Blood-pressure" in your model. Now begins the fun:

6.2.19 @12:44am - (Replying to @VDimitrakas) Willing to change adjective. "primitive"? "incomplete"? "limited" ? "insufficient"? "weak"?. Open to suggestions.

6.2.19 @12:28am - Unfortunately, what I said about statistics was misconstrued to mean "bashing" or "contempt". It was not. It was an objective assessment of what can be achieved with the impoverished language that statisticians had to work with. I hope #Bookofwhy is taken this way.

6.1.19 @10:13pm - I've never seen a book so cheap -- 2 pounds on Kindle -- a real bargain. I assumed readers lost interest, but it is still #1 in "probability and statistics" (after everything I said about statistics!) and "AI and Semantics". No explanation! Causal or otherwise #Bookofwhy,

6.1.19 @3:43am - Another supplement to Didelez etal paper on Mendelian randomization can be found in this re-posted blog discussion: ... It took place in 2014, when Bryant and Elias attempted to show Imbens how DAGs can help him live a fuller IV-life @mendel_random #Bookofwhy

5.31.19 @11:48pm - Some readers of Didelez etal paper are surprised by the absence of "d-separation" from the discussion. No need to worry; it is substituted by a surrogate called "Moralized graph", the way it was first proven in Europe by Lauritzen etal (1990)

5.31.19 @11:11pm - It all started when Nancy Pelosi elevated the bigotry of Ilahn Omar to the pulpit of the US Congress and no Democrat dared ask her to consider the consequences. I tried ... Left unchallenged hate is infectious and my fellow Democrats are now stunned helpless.

5.31.19 @12:56am - (Replying to @Basucally @edwardhkennedy) Not merely "to define", but also to derive properties that follow immediately from the definition. For example, consistency, ignorability, testability etc, See #Bookofwhy

5.31.19 @12:49am - This paper by Didelez etal is indeed a VERY good start. It is written in DAG's language and contains extensions to non-linear systems. It need be supplemented with new results on "conditional IV" and "Instrumental Sets", plus specific tasks where MR must listen to DAGs #Bookofwhy

5.30.19 @5:31am - (Replying to @mekarim7) Thanks for catching this typo. #Bookofwhy

5.30.19 @5:27am - It turned out to be more educational (for me) than I thought, and I will soon look into whether Mendelian randomization can be the next beneficiary of DAG's power. Bullet proof vests are back in the closet, awaiting a "mostly harmless" economist. #Bookofwhy

5.29.19 @4:24am - (Replying to @maximananyev @rodrikdani) I believe our conclusions are almost identical. Though I have hard time assessing the over all benefit of the "credibility revolution," as I explain here: . #Bookofwhy

5.28.19 @6:37am - (Replying to @bnielson01) The memory and traces of alternative options are missing from this definition. Does a thermostat have free-will? It seems to satisfy your "error correction" definition.

5.28.19 @4:27am - Finding yourself in this stellar company makes you really humble. What? Who? Me? You must be kidding!! #Bookofwhy

5.28.19 @2:14am - (Replying to @strangecosmos) Agree, yet moralists jump out of their wits if you ask them to accept that free will is just an illusion. This is why the ancients insisted on saying: Its not an illusion, its actually there, 'freedom of choice is always granted'. It seems to clash with determinism -- no more.

5.27.19 @11:37pm - (Replying to @zakkohane @segal_eran and 3 others) Feeling jealous being unable to be with you at the Weizmann Institute (I spent one sabbatical there) and discuss new ideas on AI/Big data with colleagues and students. Have a great meeting.

5.27.19 @11:08pm - (Replying to @strangecosmos) No difference, assuming that the organism that does not have free will has the computational facility to generate the illusion of having free will.

5.27.19 @7:56pm - It is program synthesis with an additional ingredient: the synthesizer module leaves traces in short term memory, so that it can go back to where it was before the incident, re-simulate itself, and perform the repair in accordance with the instruction. #Bookofwhy

5.27.19 @4:28pm - This is how "You should have slowed down" should be interpreted. The key point is, "I, the teacher, do not know your software, so I cant tell you exactly what to tweak, whatever it is, make sure that after tweaking, the resulting action should compute to "slow down". #Bookofwhy

5.27.19 @4:27pm - (Replying to @bnielson01) This is how "You should have slowed down" should be interpreted. The key point is, "I, the teacher, do not know your software, so I cant tell you exactly what to tweak, whatever it is, make sure that after tweaking, the resulting action should compute to "slow down". #Bookofwhy

5.27.19 @1:22pm - Fixing the program would fix the criminal robot, but will not tell others what was wrong with the offensive program. When we advertise widely "for bad behavior of type-I" an entire community of robots undergoes fixing; namely, reassignment of priorities to software #Bookofwhy

5.27.19 @1:09pm - It is the illusion of free will that allows us to supplement punishments with specific repair instructions, eg. "You should have slowed down" , as if the robot had an option to act differently. It's an effective communication trick [ Chapter 10, #Bookofwhy]

5.27.19 @2:33am - Readers remind me that this is precisely the motto quoted below the title of chapter 10 of #Bookofwhy. It reads: "All is pre-determined, yet permission is always granted." -Maimonides (Moshe Ben Maimon) (1138-1204). Note that he does not pose it as a dilemma, but as a fact.

5.27.19 1:28am - (Replying to @mendel_random @UCLA) Barring quantum noise, I am planning to be there. And if you have a list of discussion items, it will give me a chance to prepare the right bullet-proof vest. In gratitude, I will hand you a signed copy of #Bookofwhy and a personal invitation to join the "causal revolution."

5.26.19 @7:52pm - (Replying to @hlprmnky @TheAnnaGat) Godel is right! Our robot cannot have a detailed wiring diagram of its software, but this does not preclude having a blue-print of its software, which may be sufficient for conducting free-will flavored conversations, as we do with the rough blue-print of our software.#Bookofwhy

5.26.19 @6:25pm - (Replying to @TheAnnaGat) Once we understand the neural wiring behind the FW illusion there will be no FW problem. The sentence "I have an option" will be interpreted in terms of the wiring, an interpretation that precludes having an option. #Bookofwhy

5.26.19 @6:03pm - In the old days the FW dilemma was: "God can predict what you are going to do, how can he punish you for what you did?" Modernity replaced God with the equations of physics, messed with quantum uncertainties. Our problem is a robot which surely "has no FW", should we punish HIM?

5.26.19 @4:43pm - (Replying to @jrwill9) Compatibilists (like me) believe causation IS compatible with free will. The question is, does it make sense to tell a robot (driven by deterministic algorithms): "You SOB, you ought to have known better" same way as we talk a child who, presumably, does have free will.#Bookofwhy

5.26.19 @7:50am - (Replying to @chandra1250) I would not be surprised if it has. But I would love to see a reference. I did not call in "emergent property" because of interest to us is the CLASH between two levels of description, not the appearance or disappearance of a property. #Bookofwhy

5.26.19 @6:55am - (Replying to @DebyNavarroR) Zionophobia is an obsession, not a mode of thinking. The only logic it recognizes is the mechanics of slander. They are incapable of realizing that the fate of real human beings may be affected by their barking.

5.26.19 @6:32am - (Replying to @KerrShip) I dont take those attacks to be "nasty" or personal. Zionophobes truly believe they have a monopoly on truth, human rights and social justice. They are deeply shocked to find a thinking organism challenging their bubble. It was important to expose their moral blinders.

5.26.19 @6:19am - An interesting take on free-will. ... They are distorting somewhat what I say, but no harm. I say "there is no free will but it is beneficial to have this illusion and to act as through it is real". They say: "Pearl says there is free will" Not much different

5.26.19 @5:25am - (Replying to @Spinozasrose) One good thing about Zionophobic posts, they sometime link to gems of articles. I have found this gem of Amos Oz ... referenced in an article by a Zionophobic supremacist writer named Litvin. Its worth reading to empower the armies of sense and co-existence .

5.26.19 @4:35am - (Replying to @joshua_saxe @DebyNavarroR) An important clarification is needed: by "Jewish" we mean Jewish in the peoplehood sense, not religion. Otherwise the Zionophobes will jump at you with: "Aha, you want a theocratic state! Like Iran or Egypt!" Luckily the majority of Israelis are secular, bonded by heritage.

5.25.19 @10:45pm - (Replying to @droverbytrade) Are the actual slides available?

5.25.19 @9:14pm - Murray Gell-Mann (Nobel 1969) died yesterday at 89. ... I happened to meet Murray twice and, inspired by the IC algorithm, he showed keen interest in causal discovery. He was fun to be with, always curious and always offering an opinion, often blunt #Bookofwhy

5.25.19 @7:05pm - (1/3) As I tweeted earlier, I must terminate this strange discussion with enemies of co-existence who are genuinely SHOCKED to hear that some people hold different views on Zionism. This is typical of people in the self-righteous bubble of the far-far left. They see themselves as
5.25.19 @7:05pm - (2/3) God's anointed priests of human rights and anti-racism. So, when someone reminds them that other people have rights too, and that denying those right is RACISM, they undergo a traumatic mental SHOCK, "We? Racist?" they ask, "Unheard of!" Someone must tell them:
5.25.19 @7:05pm - (3/3) "Yes, look yourself in the mirror!" I hope the mirror convinces them. In the meantime, I going back to science, but not before offering readers another glimpse at "Who is indigenous?" here: ... and liberal definitions here: ...

5.25.19 @4:27pm - Seeing that you are starting to put ugly words in my mouth, I have to cut off this conversation. For the record, what I have articulated is that Zionism is a home-coming endeavor - a restoration of human rights to indigenous people: ... ...

5.25.19 @4:22pm - (Replying to @SamerAbdelnour @Vieroe @jvplive) Seeing that you are starting to put ugly words in my mouth, I have to cut off this conversation. For the record, what I have articulated is that Zionism is a home-coming endeavor - a restoration of human rights to indigenous people; ...

5.25.19 @3:59pm - (Replying to @SamerAbdelnour @Vieroe @jvplive) I dont think you read carefully what Zionism is. It is POLITE to let people who uphold an ideology define what it is, not those who labor to defile it. An ideology is defined by the way it is taught in kindergartens, not the way diplomats vote, then change their votes, then.....

5.25.19 @3:42pm - (Replying to @KerrShip) The settlement are built on wheels -- some will be uprooted, and some will remain as tolerated Jewish minority in a Palestinian State, once an agreement is reached. The obstacle is the agreement, not the settlements.

5.25.19 @3:29pm - (Replying to @jmourabarbosa) Sorry to disappoint you. But being a scientist means being honest, worship truth, and constantly tune to new evidence. It may disappoint people who have formed an opinion by reading hateful literature about Israel, but I was there, I am well informed and craving for co-existence

5.25.19 @3:12pm - (Replying to @SamerAbdelnour @Vieroe @jvplive) You are right! BDS is not anti-Semitic, it is Zionophobic, a more dangerous form of racism: See why: ... and also: Note: No BDS supporter has ever said: "I am not anti-Zionist" so, the label "Zionophobic' should be a badge of honor

5.25.19 @7:49am - (Replying to @Vieroe @SamerAbdelnour) You forgot that I am also a student of counterfactuals, so lets do a little exercise: Assume the occupation is lifted today, do we have any evidence that Palestinian's violence will subside rather than increase, given their improved position and what they promise their children?

5.25.19 @7:39am - (Replying to @SamerAbdelnour @mszargar @AsraNomani) They tell me that German's fatalities in WW-II were even more lop-sided. What does it prove? That the maximizer fails to accomplish its aims? Oh, I forgot to ask: can you now say: "equally indigenous"? In a hundred years from now? Two hundreds?

5.25.19 @7:28am - (Replying to @wgeary) That Israel tries to minimize fatalities is clear even to her enemies, otherwise they would not use civilians as human shield. After all, even if Israelis are inhuman monsters, fatalities are bad for public relation. Lets stop this nonsense, I know Israel, and you know it too.

5.25.19 @7:15am - (Replying to @SamerAbdelnour @mszargar @AsraNomani) When moral grounds are shifting we invoke "international law". Surely, no one is obligated to accept anyone's neighbor. But those who cry "oppressed" could strengthen their case by showing some commitment to a permanent peace, once oppression is ended, by agreement.

5.25.19 @6:58am - (Replying to @SamerAbdelnour @mszargar @AsraNomani) If I were to smell racism it would be in the glaring asymmetry between: "equally indigenous" on one side and "Me, Me, Me" on the other, followed again by empty slogans from BDS dictionary.

5.25.19 @6:46am - (Replying to @wgeary) Body-count says nothing about the conflict, especially when one side tries to minimize fatalities and the other brags on maximizing them. In statistics we call this a typical case of "selection bias". If you seek peace, support the minimizer, not the maximizer.

5.25.19 @6:32am - (Replying to @belial42) Never in human history was a nation threatened with extinction called an "oppressor". India never questioned the legitimacy of Great Britain, nor did Algeria threaten the sovereignty of France. Eliminationism has its price.

5.25.19 @6:21am - (Replying to @SamerAbdelnour @mszargar @AsraNomani) Calling people "settler colonialists" does not make them less indigenous than your family. Those "colonialists", remember, recite poetry written in that land, in the language spoken in that land, by heroes ruling that land. By dismissing their claims you diminish yours.

5.25.19 @6:05am - (Replying to @belial42) If I were under the boot and I knew that one word would set me free, I would say the word, even if I dont mean it. Trouble is, its hard to cheat, because if you really believe that Israel is temporary, you can't tell your child "Israell is permanent." Its tough to be eliminator.

5.25.19 @5:51am - (Replying to @Vieroe @SamerAbdelnour) Nice theory, except it does not match with facts. The besiegement started in 1936 (no occupation) when Haj Amin al Husseini declared a boycott on Jewish products, and a genocide by entrapment on European Jews seeking refuge from Hitler (My grandparents were among the entrapped).

5.25.19 @5:38am - (Replying to @KerrShip) How about one nation globally? How about just US and Mexico, share the land with equal rights to all, and happiness ever after? HMM! Now we begin to see some difficulties. Well, multiply those difficulties by 100 and we end up with "two states for two peoples". i.e., Zionism-101

5.25.19 @5:01am - (Replying to @Vieroe @SamerAbdelnour) So let us work together to allay this perception of threat. I will do it by telling my Israeli friends "It is not 150,000 rockets, it is only 130,000" and you do it by saying aloud: "equally legitimate and equally indigenous ". Do we have a deal?

5.25.19 @4:52am - (Replying to @SamerAbdelnour) All people, oppressed and not-oppressed, must accept each other right to freedom and dignity. Those who deny freedom to their neighbors must accept some responsibility for the consequences. What I hear on Twitter however is continuous one-side denial, "Me, Me, Me," not one "Us".

5.25.19 @4:35am - (Replying to @SamerAbdelnour @Vieroe) Truth can be shocking, agree. But anyone who aspires to co-existence needs to examine the state of mind of both sides, including Israelis, 95% of whom believe they are under siege by, first, rejecting neighbors, second, 150,000 Hezbolla rockets, third, Iranian proxies, more?

5.25.19 @4:22am - (Replying to @belial42) Sorry, but if you use populist slogans such as "apartheid, shooting children, bulldozing," it is hard for me to believe that you are genuinely "interested" in a comparison. It sounds like you really think Israel "shoots children" for pleasure. This is BDS thinking, tell us more.

5.25.19 @4:05am - (Replying to @Vieroe) I will describe it by, first , avoiding populist slogans (eg oppression, violation, illegal) which connote sadistic intents and, second, a temporary and unwanted predicament imposed on Israel by neighbors who openly promise her demise if she withdraws.

5.25.19 @3:50am - (Replying to @belial42) It is not a matter of "being nice". It is a matter of "as soon as they do not wish us dead, and openly say so". Some difference!

5.25.19 @3:46am - (Replying to @Vieroe) The Israeli government is a product of 71 years of beseigement under existential threats. "Give us one year of normalcy," say my Israeli colleagues "and we will show you what our government can do.

5.25.19 @3:35am - (Replying to @mszargar @NYTimesCohen) I have met literally hundreds of BDS supporters. None of them can utter the words "equally legitimate and equally indigenous". It is against their DNA. I even challenged some of them to

5.25.19 @3:28am - statehood once they grant this right to their neighbors. My friends in Israel are tuned to Palestinian schools and mosques for an INKLING of acceptance -- none thus far, just plans of elimination.

5.25.19 @3:06am - (Replying to @SamerAbdelnour @mszargar @AsraNomani) This is what BDS calls for: ...

5.25.19 @3:52am - (Replying to @SamerAbdelnour @mszargar @AsraNomani) I was right. You cannot bring yourself to say the words "equally indigenous" and you admit that SJP and the entire Palestinian leadership still deny the right of my people to a homeland in any borders. This is what I meant by "victim's cover." A cover for a plan of elimination.

5.25.19 @2:01am - (Replying to @SamerAbdelnour @mszargar @AsraNomani) The principles are simple: "Two states for two peoples, equally legitimate and equally indigenous". Your saintly "marginalized, non-violent movement" (SJP) will not be opposed if it adopts these principles. But it can't, because it was created to fight them under victim's cover.

5.25.19 @1:17am - What has BDS to do with Zionism? Everything! See ... What is Zionism? The right of a people to a homeland -- in some borders. A right denied to my people by every BDS supporter. A genocidal denial that started the conflict 72 years ago, and remains its core.

5.24.19 @7:06pm - (Replying to @causalinf @PHuenermund) I am speechless. What is the research question? assumptions? data?

5.23.19 @10:42pm - (Replying to @robertwplatt) Hard to implement. A review may be 3-5 pages, and all you want to tell the world is that seasoned reviewers in the 21st century still think that "causation is just a species of correlation" or that "confounding is a statistical concept" etc. History needs to know that. #Bookofwhy

5.23.19 @10:28pm - If Israel is so evil, why shouldn't SJP be supported? NYU campus needs to hear that President Hamilton's objections to BDS and SJP are based on moral principles and shared liberal values, not on donors or alumni pressure. Details are here: @AsraNomani

5.23.19 @9:27pm - Every time someone posts this speech I feel an urge to share it with others, finding myself re-identifying with these ideas, and with the realization that, unless re-enforced, they tend to fade into the background. I worked a lot on this commencement speech - my first. #Bookofwhy

5.23.19 @4:16pm - I propose to re-run our poll, now that we have discussed the pro and cons of publishing juicy reviews of submitted papers.
-- Is it ethical to publish selected extracts from anonymous reviews of your submitted paper?
72% yes
28% no 407 votes. 5 hours left

5.23.19 @9:03am - (1/1) This is amazing!! Thanks for finding it. I can't believe that econometric students would sit through this class and not laugh how hard the instructors sweat to avoid commonsense. I Imagine one of these students meeting an epi friend in the library, or reading #Bookofwhy .
5.23.19 @9:03am - (2/2) or hearing about d-separation, and asking: "Wow! And all this sweat just to prove that economists can do things independently and differently, home- grown style? It is still great progress for eco. education compared with PO w/o dags. See

5.23.19 @7:08am - (Replying to @JicMic) Truth and justice trump slogans and herd mentality.

5.23.19 @6:53am - (1/2) Thanks for posting this confession. I have felt the same when I had to renounce my Award. I have also come to realize that NYU administrators do not know how to handle the monster created on their watch, and no one tells them how. They dont realize that all they have to do is
5.23.19 @6:53am - (2/2) to tell the campus the reasons why Zionist students and faculty are welcome on campus, the distinct contributions they make to the cultural tapestry of NYU and the inspirational power of Israel's miracle on other minorities. They just need to be honest and tell the truth.

5.23.19 @4:09am - Sorry, typo! It is a Rung-1 task on the Ladder of Causation (see #Bookofwhy). Diagnosis goes from evidence to beliefs not caring whether the beliefs are about causes (of the evidence) or about consequences thereof.

5.23.19 @3:52am - An amazing investment in health science and data science: . I am mighty proud to see how the super-education I received in Israel is now spawning pioneering projects of such magnitude to benefit science and mankind. #Bookofwhy

5.23.19 @3:36am - The paper aims at finding minimal Boolean functions that separate positive from negative cases. A Rung-2 task. The function found explains why your diagnosis algorithm concluded that you have disease D. It does not explain why you have disease D or what would cure it. #Bookofwhy

5.23.19 @12:29am - (1/2) (Replying to @theblub @twainus and 10 others) Let me end with one more remark, to rule out any impression that I do not accept metaphysical questions as legitimate. Take the question: "what is a cause". It is legitimate if you allow me to translate it into "how can we explain the consensus among Sapiens about certain
5.23.19 @12:36am - (2/2) (Replying to @yudapearl @theblub and 11 others) cause-effect relationships. This now allows me to further translate the question into a computational one: Find a parsimonious mental representation of shared knowledge that explains how humans can access that representation and swiftly come up with same answer. #Bookofwhy

5.23.19 @12:13am - (Replying to @Kleee @DorotheaBaur) They played this role effectively until mid 20th century. They lost it when they rejected computational metaphors from entering the language of dialogue. My humble prediction: Searle and Dreyfus objections will become curious anecdotes in the history of AI and HAI. #Bookofwhy

5.22.19 @10:38pm - (1/2) (Replying to @theblub @twainus and 10 others) 1/2 I was not asking for a book or a paper, but for an idea or a method. Whether an idea is useful or not depends on what your question is. eg. Algebra is useful if my question is: what values of x satisfy certain conditions. The question "What is taste" on the other hand
5.22.19 @10:44pm - (2/2) (Replying to @yudapearl @theblub and 11 others) does not strike me as a legitimate question because I do not know what kind of answer would satisfy the questioner. Take: "taste is what unicorns swear by" or "taste is a combination of neural activation that result in a scream 'SPICY'", which would be more satisfying? Why?

5.22.19 @7:04pm - (Replying to @SiaKordestani @SFSU) No! Times are changing. It is not "anti-Semitism seeking to cloak itself as political discourse". Rather, it is Zionophobia seeking to cloak itself as normative anti-Semitism".

5.22.19 @1:30am - (Replying to @theblub @twainus and 10 others) The way I speak may betray years of frustration to understand this literature. But if you examine my work your will see that I have never turned away a useful idea because of a mind made up (at least I do not recall such mishap). What's the idea I cannot afford to miss?

5.22.19 @1:17am - (Replying to @Plinz) Everything could be part of higher education if so presented. But accusing you of child molestation in a class on human sexuality is not exactly conducive to higher education. Abdulhadi did precisely that, and we, normative Zionists, now request to be publically decriminalized.

5.22.19 @12:56am - (Replying to @theblub @twainus and 10 others) My language is: English, math, physics, programs and human cognition. Using this languages please tell me one idea that I should learn from Dreyfus , without which I am going to waste lots of time on dead ends. No ref to other philosophers please, no acronyms nor Heidegger

5.22.19 @12:17am - (Replying to @theblub @twainus and 10 others) I am dying to learn from them, but I have not found anyone who can translate them to a language I understand. In particular, to convey an idea from beginning to end without quoting five other phenomenological philosophers, so as to make you feel an outsider. Substance please.

5.22.19 @12:08am - (Replying to @Plinz) Naive it may be, but it need to be aspired to, and if no one says the word "racist" students may get the idea that Adulhadi is a normal person, perhaps even an "educator", so perhaps I and my Zionist colleagues are indeed "white supremacists" . Sorry. She is just a Zionophobe.

5.21.19 @11:47pm - (Replying to @Plinz) It is not "sacralization" but common decency. There is such a term in English called "racism", which is sometimes used improperly and sometimes properly. University administrators often condemn hate speeches that take place on their watch, to set the norms right. No sacralization

5.21.19 @11:28pm - (1/2) Readers ask if I was not too harsh calling Abdulhadi "racist". My answer: What would you call a guest lecturer who comes to your university saying "Muslims are terrorists, but I have nothing against Muslims who disavow Mohammad". Evidently, archaeology professors think it's
5.21.19 @11:28pm - (2/2) "educational". The fault is not really with the instructor, but with a university that does not internalize the equation: Zionophobia = Islamophobia, which means that religion does not have a monopoly on human identity, and all symbols of identity should be equally respected.

5.21.19 @10:56pm - The Dreyfus video reminds me of Marvin Minsky who once said: When I hear someone saying: "A computer cannot do Y", what he is really trying to say is: "I havn't got a clue what Y is". I will change my mind if I someone translates Dreyfus into English, math or program. #Bookofwhy

5.21.19 @6:19pm - Replying to @twainus @yhazony and 9 others Any idea when this video was taken? Dreyfus is repeating the same ancient arguments that I heard him state in the 1970's. Is he back in fashion? #Bookofwhy

5.21.19 @3:40am - This is amazing! So access to anonymous reviews IS available from some journals. Victory!! I feel less inhibited to start my juicy selection. #Bookofwhy

5.21.19 @12:08am - (1/3) Our poll shows a slight preference - 58/42 - to allowing publication of juicy quotes from anonymous reviews. I have hoped for a more decisive preference. Aside from the entertaining value of such a collection, and its encouraging effects on young researchers,
5.21.19 @12:08am - (2/3) I am concerned with its historical value. Written under the shield of power and impunity reviewers comments are the most honest and faithful reflections of the state of mind of a scientific community at any given period. Such historical treasure should not be allowed to rot
5.21.19 @12:08am - (3/3) in the archives of outdated journals. There should be at least some status of limitation before unveiling this information to the public. Anyone knows what happens to these archives? Can a historian request access to the reviews of Turing's paper of 1937 ?? #Bookofwhy

5.20.19 @6:57pm - For all who requested a copy of my contribution to the book "possible minds" (edited by J. Brockman, 2019), I now have a link to it: So let us continue our voyage, from Babylon to Athens. #Bookofwhy

5.20.19 @4:02pm - (Replying to @68kirk) Judea Pearl Retweeted Judea Pearl Causality is all about invariance, of course. But the relationships between these and machine learning should be articulated this way:

5.20.19 @1:47pm - My! My! Thanks for posting this picture --it made my day. I have enjoyed telling students my personal story on surviving 1948. Too bad I did not meet Miss Iraq, my wife is from Baghdad. Go on spreading the word about an inspiration called Israel. Happy Birthday!

5.20.19 @1:32pm - (Replying to @adviceonstock @IanCero) I have a strong issue with this statement. Machine Learning and Deep Learning models do NOTsupport casual inference, unless by "support" one means "can be used in", like "arithmetic supports classical mechanics." See #Bookofwhy

5.20.19 @3:00am - (Replying to @glarange72) CI is Causal Inference, and by "CI + ML hype" I mean the noise generated by the many who claim to be combining the two when they can barely do one. #Bookofwhy

5.19.19 @10:57pm - If you are inspired, as I am, by new perspectives on causal data-science, marinated in substance and free of CI+ML hype, you will enjoy this lecture by Elias Bareinboim which was delivered at Columbia last month Recommended to readers of #Bookofwhy

5.18.19 @8:25pm - (Replying to @maximananyev) Terrific PhD topic: Write a program that automatically generates such a news article. Input: voting data + polls + voters model. Output: a post-election article on "why the polls got it so wrong". Criterion for PhD: article accepted for publication in a top newspaper. #Bookofwhy

5.18.19 @10:45pm - (1/2)
For many years now I have been playing with the idea of publishing a selection of juicy reviews that my papers have received from anonymous, self-assured reviewers. Some colleagues advised against it, since it violates reviewers' trust in eternal concealment. Here is a poll
5.18.19 @10:45pm - (2/2)
1. As a reviewer, would you mind finding portions of your review in print (anonymously, of course)?
2. As an author, do you think this would improve quality and accountability of reviewers?
33%(1) yes; (2) yes.
09%(1) yes; (2) no.
41%(1) no; (2) yes.
17%(1) no; (2) no.

5.18.19 @6:47pm - (Replying to @aesopesque) I do not think risk-averseness would change any of the results; we can always assume that benefits are measured in "utiles" instead of "dollars". #Bookofwhy

5.18.19 @4:23pm - Good luck for the 2020 Eurovision, and happy singing. From Israel (2019), with love, to The Netherlands (2020).

5.18.19 @1:14am - How do you select customers (or patients, or voters) that are most likely to respond to your request/action when "responsiveness" is an unobserved, counterfactual notion? A new paper by Ang Li proposes a method. #Bookofwhy

5.15.19 @11:43pm - (Replying to @WLoosen @SZ) @Wloosen, Thanks for posting. Funny, I received the Dutch translation of #Bookofwhy in the mail today, and could not tell if they mean it. And saintly Mother Theresa on this article made me fall in love again. I love Dutch, the language of Huygens and Till Eullenspiegel

5.15.19 @10:10pm - When I first posted this birthday note, readers asked: Who are the 1948-deniers? Today we have a super-denier in Congress, Rashida Tlaib, who never heard Azam Pasha call for "a war of extermination and momentous massacre" against Israel; Sec. Gen of Arab League, Oct. 11, 1947

5.15.19 @7:46pm - Carlos etal have a new paper on sensitivity analysis which, to the best of my knowledge, is the first to recruit graphical models into this struggling enterprise. I expect to see soul-searching among traditionalists and awakening in the field. #Bookofwhy

5.15.19 @3:38pm - (Replying to @AndersHuitfeldt) How about this:

5.15.19 @2:02am - I was sent a new article on Evidence-Based policy. ... Perhaps it can enlighten readers to tell us where this enterprise stands in the microcosmos of causal inference. #Bookofwhy

5.15.19 @12:38am - (Replying to @DickeySingh) Sure! All we need now is to understand how you decide that the pressure causes the barometer reading, not the other way around. It may be possible, but to understand what's behind it, we need an explanation cast in conditional probabilities, not in training algorithms.

5.14.19 @11:27pm - Kudos to Maayan Harel @maayanvisuals for reminding us of the #Bookofwhy anniversary, and for inspiring us with her eye-opening illustrations. May 15 is also the general calendar birthday of Israel, Maayan's (and my) birthplace - We have much to celebrate tomorrow. Ko Lechai

5.14.19 @5:38pm - (Replying to @mrgunn @vgr @StatModeling) 1.8.19 @11:59pm -- Gelman's review of #Bookofwhy should be of interest because it represents an attitude that paralyzes wide circles of statistical researchers. My reaction is now posted on Related posts: and

5.13.19 @2:45am - (Replying to @HenningStrandin) Correct. But this does rule out "probabilities of counterfactuals" since those individuals may have different behaviors, with a distribution over the behavior types.

5.13.19 @2:15am - (Replying to @eadeli) Wait, wait, you did not get to the part where I encourage students to rebel against their textbooks. This will surely make you angry. #Bookofwhy

5.13.19 @1:58am - (1/ ) I was delighted to read your Causal Ladder blog-post, especially the way you explain the necessity of the counterfactual layer and the vivid examples you used. (I literally forgot the great party we had with Ann and Bob). A word about the exogenous variables U:
5.13.19 @1:58am - (2/2) These variables specify a "unit", be it an individual, an agricultural plot, time of day, etc, whatever refinement is needed to make all relationships deterministic. I hope this clarifies the dilemma posed in your last paragraph. #Bookofwhy

5.12.19 @3:39pm - I must fully endorse this recommendation . I did not realize how much I owe to my training in physics (eg., ...) until I had to face the model-blind thinking of modern statistics, and the difference between statistical and causal models. #Bookofwhy

5.12.19 @2:40pm - (1/ ) Communication between CI and ML folks will improve drastically if we can translate sentences such as: "Bottou trains his NN under conditions ABC" into sentences of the form: "Given the conditional probabilities P(y|x,do(z)...)". After all, what do we get from "training" if ..
5.12.19 @2:40pm - (2/ ) if not conditional probabilities, both observational and interventional. Another benefit for the translation: theorems of impossibility. CI has developed a theory that tells us if certain tasks can be accomplished given information in the form of probabilities P(y|x, do(z)...
5.12.19 @2:40pm - (3/3) We can use this theory to prevent disappointments from "training" schemes that lead to impossibilities. As far as I know, theories of what's possible or not possible were not developed (yet) for training schemes. Why not use what we have? eg, #Bookofwhy

5.11.19 @3:32pm - (Replying to @isli_amar) Your proposal is sincere and well-intended, but it crumbles against the logic of Algerian children who are asking: If Israel is a colonial endeavor (like French rule in Algeria) why should it be recognized in ANY border? Indeed, WHY? Are you prepared to tell them the truth?

5.11.19 @1:48pm - (Replying to @_fernando_rosas @KordingLab) Terrific question!! (which every student want, but is afraid to ask). Answer: We learn quantitative "effect size", P(y|do(x)), while before we had only qualitative information, eg. "X does not listen to Y", or P(x|do(y))=P(x). #Bookofwhy

5.11.19 @1:41pm - (Replying to @nlpnyc) "1948-deniers" are authors (eg Gelvin) who deny that the 5-army attack on Israel in 1948 was GENOCIDAL IN INTENT: "a war of extermination and momentous massacre which will be spoken of like the Mongolian massacre and the Crusades" (Azam Pasha, Sec.Gen. Arab League, Oct. 11, 1947)

5.11.19 @1:18pm - (Replying to @_fernando_rosas @shell_ki) The observed data is just "observed data" ie, a bunch of correlations among variables. It is generated by causal relationships, yes, but it does not tell us what those relationships are, unless we assume extra-distributional assumptions (eg graph) called "causal model" #Bookofwhy

5.11.19 @3:32am - (Replying to @isli_amar) Yes. I remember 1967. The world was waiting for a plausible peace process when, on August 29, the Khartoum Arab League Summit issued its famous: "Three No's"; No peace with Israel, no recognition of Israel, no negotiations with Israel. They can still reverse it-- we are waiting!

5.11.19 @12:26am - Had a great celebration yesterday of Israel's Independence Day - speaking to students as a "1948 eyewitness", feeling like an endangered species, and thinking gloomily: who will bear witness when my generation is gone and the professional 1948-deniers take over?

5.10.19 @11:46pm - When I was a student, speaking against #racism was an obligation, not "courage." It's not funny, the change happened on our watch!!

5.10.19 @2:28pm - (Replying to @sweichwald @atypical_me and 3 others) Your definition is a good one. But I would like to embrace partially specified models under the label "causal model" . For example: X----->Y + X<---U--->Y which does not identify P(Y|do(X)) is still a causal model; it tells us P(X|do(Y)) = P(X) #Bookofwhy

5.10.19 @1:40am - (Replying to @eigenhector @ericjang11 and 8 others) In CI, we call it "functional model" or "completely specified structural model", where the response of each listening variable is functionally defined. In such a model all counterfactual queries, conditional as well as marginals, are estimable. Why invoke quantum? #Bookofwhy

5.10.19 @12:57am - (1/ ) (Replying to @ericjang11 @eigenhector and 8 others) In CI we classify problems along 3 dimensions: 1. what we know, 2. what we wish to know, 3. what type of data sources we have. We then ask: Can we obtain (2) from (1)&(2)? We try to avoid giving agency to algorithms and acronyms. For example, "Model-based RL" is too vague for
5.10.19 @1:03am - (2/ ) (Replying to @yudapearl @ericjang11 and 9 others) evaluation, because its capabilities depends on the kind of "models" invoked, and because you can do everything with a highly refined "model". Whether RL and its varieties can accomplish one task or another depends on (1) and (2), see ... The best way to
5.10.19 @1:12am - (3/ ) (Replying to @yudapearl @ericjang11 and 9 others) communicate about capabilities is to use canonical examples. For example, how would acronym ACR handle the napkin problem? or Joe's "would be salary" problem? Is the output guaranteed to be consistent? In these canonical examples, (1) and (3) are formally specified,
5.10.19 @1:17am - (4/4) (Replying to @yudapearl @ericjang11 and 9 others) as in , leaving no room for ambiguity, thus enabling us to determine which query can be answered from a given combination of knowledge and data #Bookofwhy

5.9.19 @10:50am - (Replying to @theAlexLavin) Great way to think, agree. But what principle need we assume to show that y=f(x,e) is more "invariant" than say x=g(y,e').

5.9.19 @10:40am - Not so. Not when two sides agree to a win/win sharing partnership. Besides, we have not heard this equation on Cinco de Mayo, last week. Reminds me of a parody I wrote on editors who feel the urge to spoil birthdays with equations: Happy Birthday Israel!!

5.9.19 @2:45am - (Replying to @PHuenermund) My, My! I did not realize Israel shares a birthday with EU, both determined to prevent another genocide, and both succeeding thus far, to some extent. I wish however that Israel's peace would depend on something so tangible as coal trade.

5.9.19 @2:31am - (Replying to @BreyonWilliams) Congratulations, Breyon. I usually add: well deserved, after reading one's dissertation but, in your case, I am willing to bet it is. So, welcome to PhD-land, and I hope you revolutionize econometrics.

5.9.19 @1:23am - Today, May 9, is Israel's Independence Day. I invite all readers to join me in celebrating the 71st birthday of the country where I learned to speak, and thanking her for what she has contributed to mankind, and for redefining the meaning of "miracle." Happy Birthday Israel !!!!

5.9.19 @12:14am - (Replying to @arih1987 @Stanford and 2 others) The trick is not to dis-invite racist speakers but to politely explain to them why they should dis-invite themselves, as I tried to do with Cornell West ... The host, the audience and the public got the idea, and some say the speaker got it too.

5.8.19 @10:41pm - (Replying to @miquelporta @socestadistica and 5 others) @miquelprta, what brings you to mention the NYT review of #Bookofwhy? True, it is one of the best written reviews, but how is it related to p-values and to other topics discussed there, on Mt. Olympus?

5.8.19 @4:36pm - (Replying to @AngeloDalli @ylecun) A free (signed) copy of @Bookofwhy to the first person who extracts the principle from the equations.

5.8.19 @3:36pm - (Replying to @babeheim @MPI_EVA_Leipzig) I vow for all equations, Keep us informed when you dig into counterfactuals. Have fun!

5.8.19 @3:28pm - (Replying to @ach3d) good choice!!

5.8.19 @3:27pm - (Replying to @AngeloDalli) Did you get the principle? I can't get to the paper itself, can you? Invariance is always at the heart of causation,; do we have a new method of interrogating the invariance? Let me know if you dig it.

5.8.19 @3:15pm - (Replying to @NevinClimenhaga) Interesting abstract but it hides the principle/assumption till the thick of the paper. Can you summarize it for us in Tweeter length? As I tried to do here , Tool # 7: causal discovery. #Bookofwhy

5.8.19 @4:39am - (Replying to @JDHaltigan @juli_schuess @doinkboy) I sure do, but not as a stand-alone argument. Why? Because to model-blind researchers "confounding" is just "nonignorability" which is defined relative to a "treatment," and requires no notion of "cause". See how Imbens and Rubin labor to explain "unconfoundedness." #Bookofwhy

5.8.19 @2:21am - (1/ ) Depressing to see a once friendly campus (UCSD) consumed in slander. As I note here: ... -"in the grand opera of BDS's slander machine, it is not the libretto that matters but the stage and the megaphone. The charges may vary from season to season,
5.8.19 @2:27am - (2/ ) the authors may ' rotate, and it matters not whether a resolution passes or fails, nor whether it is condemned or hailed. The victory lies in having a stage, a microphone, and a finger pointing at Israel saying, "On trial!" It is only a matter of time before innocent students
5.8.19 @2:27am - (3/3) mostly the gullible and uninformed, will start chanting, "On trial!" It worked in Munich, and it has worked on some campuses. The effect will be felt among the next generation of policy makers.

5.7.19 @11:20pm - (Replying to @JDHaltigan @juli_schuess @doinkboy) I love this argument, but I dont think manipulationists like Hernan will buy it. They would say: OK,"U causes Y" colloquially, but thou shall never say "causal effect of U." Sounds a bit inconsistent? Agree. But we have not heard from them since #Bookofwhy

5.7.19 @6:51pm - Faculty of New York University: Oppose Academic Boycott of NYU Tel Aviv - Sign the Petition! via @Change

5.7.19 @6:33pm - (Replying to @djinnome @juli_schuess @sweichwald) Thanks for noting the break. We will fix it soon (I hope).

5.7.19 @1:47pm - (Replying to @martin_garcia_a @_MiguelHernan) Fine, but I am questioning the benefit of separating "description" from "prediction", skipping "diagnosis" and lumping together "intervening" and "retrospecting" under one opaque category "causal inference".

5.6.19 @8:56pm - (Replying to @thehuntinghouse @IsraelCampus and 6 others) So is the racist mentality of BDS cronies, an extremely interesting object of academic study. A newly evolved aberration.

5.6.19 @8:31pm - As I wrote to NYU president Sexton here: ... "When a group of self-appointed vigilantes empowers itself with a moral authority to incriminate the academic activities of their colleagues, we are seeing the end of academia..."

5.6.19 @8:17pm - I was happy to read today that a petition to restore commonsense to NYU has gathered over 3,000 faculty, students and alumni signatures. ... Refreshed by this response, I feel proud again being a NYU alum.

5.6.19 @1:47pm - (Replying to @ehud) Worrall lives in the pre-causal era (ie, probabilistic causality) and so, it does not satisfy my curiosity: What is "evidence-based-Med"? Is it a QUEST for principles, or a SET of principles? If the latter, is there a simple example where I NEED one such principle?.#Bookofwhy

5.6.19 @4:07am - (Replying to @XiXiDu @wtgowers) We have to take into account that some observers, even in academia, have not watched the "peaceful demonstrators" in action, and truly believe that Israelis shoot civilians for sport. We tend to underestimate the power of Hamas propaganda because it sounds so absurd, but it works

5.6.19 @3:38am - (Replying to @wtgowers) De-legitimization is a very good predictor of how one would act given the chance. The fact it has not diminished over the past 83 years means that one is very very serious about acting, given the chance.

5.6.19 @1:31am - (Replying to @ArashBroumand) You keep mentioning "politics" and "rhetorics" where I see none. I see humanity and I see human rights and I see a conflict that can be resolved when each side acknowledges the human rights of the other. This makes me an optimist, because I have found one such side already.

5.6.19 @12:58am - (Replying to @ArashBroumand) I wish I could share your optimism. Unfortunately, I know that peace can only prevail if both sides agree to the "equally indigenous" principle. I also know that the vast majority of my Israeli friends agree to it - curious what you hear from your Palestinian friends. Optimist?

5.5.19 @11:30pm - (1/ ) (Replying to @mohomran) "Being in a position" is an interpretation. The fact is that Palestinians have been denying Israel's right to exist RELENTLESSLY, for 71 years, 24/7. From school-teachers to TV anchors from journalists to imams from academicians to intellectuals. These are the people whom my
5.5.19 @11:48pm - (2/2) (Replying to @yudapearl @mohomran) friends in the Israeli peace camp are tuned to. None came forth with the word "coexistence", or "equally indigenous". Why 71? It is now 83 years of Arab "me, me, me!" rejectionism. Wait!, how about an intellectual like yourself, can you say the words "equally indigenous"?

5.5.19 @9:53pm - (Replying to @Doc_Yemen @aynumazi) We both know that Hamas/Rashida want more than just "basic necessities". We both read the Hamas Charter (Israel destruction), and we know that Rashida can never say the word "co-existence." We also know that Gazan can have all necessities were it not for what Hamas/Rashida want.

5.5.19 @8:34pm - (Replying to @Doc_Yemen @aynumazi) I was there, in that counterfactual world. Jerusalem was caged in 1947, no food no water and we were promised "momentous massacre" by the Arab League. Still, all we asked our neighbors (+ UN) was a legitimate co-existence for two indigenous peoples. Not denial of the other!!!

5.5.19 @7:08pm - (Replying to @aynumazi) I am VERY serious, and I know something about DAGs and about the logic of cause and effect. So you are invited to join me in examining the logic of Goliath/Rashida/Hamas as they deny their neighbor the very rights they demand for themselves -- freedom and self-determination.

5.5.19 @6:08pm - When will the world learn that Palestinians' conception of "freedom" is somewhat different than the ordinary. It entails freedom to de-legitimize the freedom of their neighbors.. A strange conception indeed, but some think it nevertheless deserves the Hall of US Congress.

5.5.19 @5:48pm - Goliath are those who deny their neighbors what they demand for themselves: right to self-determination.

5.5.19 @4:54pm - When will the world stop dehumanizing Goliath who just want to be free, and who just happened to forget that David and his shepherd brothers likewise, just want a day for freedom.

5.5.19 @2:06pm - (Replying to @nathansttt @ehud) Reading your wife's overview I get a strong suspicion that EBM is an emotional appeal for principles by which we can integrate epidemiological studies with informal opinions of individual physicians. Has it advanced beyond the appeal?

5.5.19 @1:53pm - The more I look at Hernan's classification of data-science: {description, prediction and #causalinference} the more I prefer the Ladder of Causation {Association, Intervention and Counterfactuals} as in #Bookofwhy. Perhaps b/c #causalinference is becoming "what everyone is doing"

5.5.19 @12:57am - (Replying to @ehud) I have never been able to figure out what "evidence based medicine" is all about. Perhaps an expert can explain? Is there a Medicine not based on evidence? Are there requirements on the methods? #Bookofwhy

5.4.19 @10:21pm - I totally agree with @_miguelHernan that the competition devised by DORIE et al. (2019) to compare "methods for causal inference" provides no information on "methods for causal infererence" and should have been titled "methods for estimating certain formulas" #Bookofwhy

5.4.19 @9:27pm - (Replying to @jim_adler @Toyota_AI_VC @PitchBook) I am trying to understand the task that the CAT is asked to perform. Anyone can explain to the uninitiated?

5.4.19 @9:23pm - (Replying to @joaoeira @gwern @Jabaluck) We tried a blog-based discussion on the same issues with Guido Imbens, see ... -- to no avail. I actually prefer Tweeting because it forces you to cut the baloney. #Bookofwhy

5.4.19 @7:58pm - (Replying to @MariaGlymour @danielwestreich @UCSF_Epibiostat) @MariaGlymour You say: "If effect modifiers are differentially distributed, trial effect estimates won't match the target population effects." Is this a new complication that is not captured by the standard conditions of or ?

5.4.19 @7:34pm - (Replying to @chophshiy) Anyone doing causation outside "conventionally qualifying institution" deserves a free copy of #Bookofwhy. If you can drop by my office at UCLA it will be waiting for you. [My sec. is on vacation this week, try next one]

5.3.19 @2:50pm - (Replying to @_julesh_) Not clear to me why the last item is "theory" instead of "counterfactuals". The first two items specify what we can do, not how we do it, shouldn't the 3rd also tell us what we can do with the "theory" that we cannot do with the other two.?? #Bookofwhy

5.2.19 @10:47pm - Confession: the juicy stuff in #Bookofwhy was partly inspired by the juicy stuff of modern academia and its submission to egos and clans. Funny, graphs are still prohibited in certain departments, and top PO researchers can't do the homework problems in

5.2.19 @7:14pm - (1/ ) (Replying to @IsraelCampus @nyuniversity and 5 others) This is a perfect time for President Hamilton of NYU to defend the Study Abroad program on moral grounds and expose the hypocrisy of the department of Social and Cultural Analysis by stating: "A country whose existence is under daily threats cannot be expected to invite in
5.2.19 @7:16pm - (2/2) (Replying to @yudapearl @IsraelCampus and 6 others) people who openly seek its destruction. I [Hamilton] advise members of the SCA department to spend their energy in support Israel's right to exist before criticizing her protective laws or policies."

5.2.19 @2:32am - (Replying to @omaclaren @bjh_ip) This distinct notation is useful, because it allows us to distinguish "; theta)" from "|X=x)". In the former, theta can be an arbitrary index; in the latter, X=x must be an event in our probability space. #Bookofwhy

5.2.19 @12:50am - (Replying to @bjh_ip) Relatedly, most modern Bayesians define their craft as that of assigning priors to parameters and computing their posteriors. Rarely do they examine Bayes' original dilemma: How do we express probabilities that we WISH to estimate in terms of those we CAN estimate. #Bookofwhy

5.1.19 @4:09pm - (Replying to @bjh_ip) Modern Bayesians would do well to take a second look at Bayes' original paper, this time from a causal inference perspective, and follow the evolution of the epistemological term: "Given that we know". I believe Chapter 3 of #Bookofwhy does a good job of presenting this idea.

5.1.19 @2:48am - (1/2) (Replying to @AlexMGeisler @clibassi) Such a course is urgently needed for data scientists as well. The reason I am calling attention to the foundations of counterfactuals is that I see even noted champions of @causalinference frequently abandoning those foundations. For example, the principles discussed here
5.1.19 @2:53am - (2/2) (Replying to @yudapearl @AlexMGeisler and 2 others) are hardly recognized by "experimental" economists, or decorated integrators of ML and CI, and I am still not sure about Harvard epidemiologists. Supreme Courts need clarity, cohesion and consensus. #Bookofwhy

4.30.19 @6:40pm - While debating Gorshuch on how causes and #counterfactuals DONT WORK, let's take a look at how they DO WORK, and what it takes to extract them from regression. Here is a gentle introduction, as harmless as they come: . #causalinference, #Bookofwhy

4.29.19 @6:53am - (Replying to @yoavrubin) Thanks, but don't forget the Primer , which stands between #Bookofwhy and Causality, and which I continue to recommend for people who want to DO #causalinf rather than talk ABOUT it.

4.28.19 @4:28pm - (Replying to @causalinf @agoodmanbacon) Mistakes like this can make history. The more I hear of such mistakes, the more I am tempted to forgo modesty and recommend this book honestly and in the strongest possible terms: Read it! For fun and insight! Here is Chapter 4 for a bait #Bookofwhy

4.28.19 @2:26am - (1/ ) I have decided to retweet my last reply because the distinction between the methodological "science of adjustment" and the substantive "science of diseases" seems to be vulnerable to ongoing confusion. Moreover, one cannot over-emphasize the miracle of the former. Not only are
4.28.19 @2:26am - (2/2) the assumptions qualitative, they are also meaningful and natural, namely, judgments about what variables determine the value of another are the easiest ones for a domain expert to articulate and communicate. Conclusions: adjustment is not an "art," it is a science.#Bookofwhy

4.28.19 @1:51am - (Replying to @EngineerDiet @TuckerGoodrich and 4 others) It is healthy to separate the "science of adjustment" from the "science of diseases". The former is valid whenever your assumptions about the former are valid. Moreover, and miraculously, the assumptions needed are qualitative: who affects whom. The rest is algebra. #Bookofwhy

4.27.19 @3:07am - (1/ ) (Replying to @sifogrante1) You are probably right about the effectiveness of MOOC format. However, we lack the administrative/institutional support needed to launch it. All we have is truth and commonsense, and we trust those who audit MOOCs to compare what they learn to modern ways of doing things,
4.27.19 @3:15am - (2/2) (Replying to @yudapearl @sifogrante1) and share their experience with us on Twitter. In particular, check if the tools taught are compatible with CI as defined in ... or, better yet, if they permit you to solve the toy problems of - my ultimate litmus test. #Bookofwhy

4.26.19 @4:10am - I was sent this illuminating talk by David Gross (Nobel, Physics) on the meaning of "truth" and the scientific method. ... The first part classifies scientific questions as in the Ladder of Causation, and should be recognized by readers of #Bookofwhy.

4.26.19 @4:02am - (Replying to @a40ruhr) Bidirected arrows do not create cycles, the simply state that the correlation is created by some hidden common cause. Still, I would turn this arrow into unidirectional and pose it as a homework for IV enthusiasts. #Bookofwhy

4.26.19 @3:45am - (Replying to @a40ruhr @USF_Economics and 2 others) I fail to detect cyclicity. Can you point to it? I think the structure is ideal to serve as a homework in modern econ. textbooks, asking students to identify the causal effect using IV method. This should lure "experimental economists" to catch up with modernity. #Bookofwhy

4.25.19 @11:17pm - I am intrigued by your DAG, can you flesh it out so we can see the labels? It seems to demonstrate that, contrary to Yale's economists, even simple IV cases need DAGs for identification. IOW, I dont see how Yale students would identify it using "mostly harmless" p.85 #Bookofwhy

4.25.19 @7:48pm - The article you posted realizes that data analysts need to: "understand the data from a wholly new perspective". Fortunately, this realization is taking place while the field is still young, so re-thinking will be less traumatic than, say, statistics or economics. Hope!#Bookofwhy

4.25.19 @1:24pm - (Replying to @Soroush_Saghaf @JoshuaSGoodman @oziadias) Eager to learn from your teaching experience. What, in you opinion, was the most useful tool, or concept, that econ. students learned from #causalinference, which they could not get from econ. textbooks, say Angrist or Wooldwridge. #Bookofwhy

4.25.19 @3:10am - (Replying to @HenningStrandin) It is definitely non-manipulationist, and that is why Rubin&Co +Harvard&Co refuse to use it, despite the fact that it gives meaning to counterfactuals, which they do use, but only when enslaved to a "TREATMENT" ; they are missing out on "undoing of past events. " #Bookofwhy

4.25.19 @2:56am - (Replying to @yskout) The beauty of this formalism is that, for certain questions (eg interventions), all you need is the graph structure; for others (eg retrospective counterfactuals) we need the functional form as well. Its a miracle, dont miss. See #Bookofwhy

4.25.19 @12:41am - Here is a more intuitive definition of Structural Causal Model (SCM). It is a society of LISTENING variables, and a specification of how each variable would respond to what it HEARS from the neighbors. Though somewhat informal, I find it more meaningful than the formal.#Bookofwhy

4.24.19 @5:19am - (1/ ) (Replying to @totteh) Agree with the wisdom, and agree with the pioneering value of this book. However, one weakness is that it does NOT give an example where POs and DAGs are used to describe the same causal structure. More severely, POs and DAGs are treated as two separate frameworks. It would be
4.24.19 @5:30am - (2/2) (Replying to @yudapearl @totteh) helpful to show readers that the two emerge from the SAME mathematical object, Structural Causal Model, as is shown for example in Section 4.2 of , especially "The Fundamental Law of Counterfactuals" Eq. (4.5). #Bookofwhy

4.24.19 @5:06am - (Replying to @nagpalchirag) If you show me one example where identifiability is easier to see in PO, I will convert to voodoo. #Bookofwhy.

4.23.19 @9:20pm - Sad news. Nils Nilsson passed away An AI pioneer, and a mentor to many of us since the 1970's. Always encouraging and always insisting on understanding new ideas, and how they fit together in the grand scheme. I will miss him immensely.

4.23.19 @8:58pm - (Replying to @nagpalchirag @DaphneKoller) First time I hear someone saying so. Could you share with us the first place where you found PO illuminating do-calculus. Eager to learn. #Bookofwhy

4.23.19 @4:53am - (Replying to @dlmillimet @PHuenermund @causalinf) Personal choices -- I will salute to that!! As long as we know what the options are. And, to accelerate progress, we can let our students too see what the options are. #Bookofwhy

4.23.19 @3:20am - (Replying to @DorotheaBaur @bobehayes) Do you think Kalev Leetaru (Contributor) read #Bookofwhy? Worth checking.

4.23.19 @3:09am - (Replying to @ingorohlfing @DToshkov) The do-calculus is "type"-level, and its relation to Woodward's interventions is discussed here:

4.23.19 @3:01am - (Replying to @DToshkov) "Intervention" is type level, eg. careless driving causes accidents. Counterfactuals are 'token' level. eg "this accident would not have occurred had you driven carefully." Both are handled harmoniously in the SCM framework, for philosophers to rejoice #Bookofwhy

4.23.19 @2:49am - (Replying to @smhall97 @alienelf @elanvb) This is a beautiful paradox, thanks for reminding me. In my former life I solved it using Bayesian analysis. Today I would approach it as a causal problem and model the process by which the envelops are stuffed; the prob. of doubling A is not 1/2 but varies with A. #Bookofwhy

4.23.19 @12:08am - (Replying to @PHuenermund @dlmillimet @causalinf) Cruel? Please compare to modern treatments of measurement error here: and here , in both clarity and generality. Cruel??? #Bookofwhy

4.22.19 @11:02pm - (Replying to @UusitaloLaura @KirsiNorros @teemu_roos) I have not seen #Bookofwhy in a picture frame before. Glad it was translated into Finnish. Happy voyage to you.

4.22.19 @9:51pm - (Replying to @jkrt168t) What makes experimental economists strangers in CI-land is the fact that they do not start with the available causal information and use it (with data) to answer causal questions. Instead, they start by applying a pre-canned procedure to the data and then ask: Does the estimate
4.22.19 @10:55pm - (2/2) (Replying to @yudapearl @jkrt168t) obtained have "causal interpretation?". Or, "Under what conditions can we interpret the estimate obtained as a valid answer to our question?" Those conditions are then articulated in opaque language (eg ignorability) far removed from ordinary scientific thinking. #Bookofwhy

4.22.19 @4:26am - (Replying to @RevDocGabriel) I am not familiar with multi-omics data.But I can refer you to integrating data from heterogenous sources involving, for example, diverse populations, diverse samples, experimental and observational data etc. See or #Bookofwhy chapter 10, pp 350-358.

4.21.19 @11:06pm - (1/2) For AI researchers still immersed in the debate between model-based vs. model-blind AI, I am retweeting a response of @wellingmax to Sutton's blog. Max agrees by and large with "The Seven Tools of CI", , albeit w/o noting the theoretical impediments.
4.21.19 @11:06pm - (2/2) Along a similar vein, I was asked to retweet the last line of my slides in the Why-19 symposium . Gladly; it reads: "Only by taking models seriously we can learn when they are not needed." And I still vow for it. #Bookofwhy

4.21.19 @8:14pm - (1/ ) In view of persistent ambiguities regarding the definition of "causal inference" (CI) I am sharing here the definition that has guided me successfully throughout my journeys. CI is a method that takes data from various sources, as well as extra-data information, and produces
4.21.19 @8:14pm - (2/ ) answers to questions of two types (1) the effects of pending interventions and (2) the effects of hypothetical undoing of past events. See Causality (2000) Chapter 1. A vivid and recurrent example of a non-causal question is any question that can be answered from the joint
4.21.19 @8:14pm - (3/ ) probability distribution of observed variables, eg, correlation, partial regression, Granger causality, weak and strong endogeneity (EHR 1983) etc. See . This definition excludes Pearson's (1911) and Fisher's (1925) descriptions of statistical tasks
4.21.19 @8:14pm - (4/4) and I would reserve judgment on how "experimental economists" fit into this definition. I believe that, in due time, "experimental economists" will manage to articulate formally what "extra-data information" they use, and thus become bonified members of CI. #Bookofwhy

4.21.19 @3:16pm - (Replying to @ahmaurya) The vast majority of economists that I know would be offended if labeled "regression analysts", in the same way that physicists would be offended if labeled "arithmeticians", though they use arithmetical operations every hour of the day. Regressionists live and die w/o causes.

4.21.19 @1:45pm - (1/ ) (Replying to @ahmaurya) You do not understand me correctly. The economentric literature is motivated by causal questions, and has pioneered modern causal inference. See . Traditional regression analysts, however, shun causation, which evoked my surprise at the paper discussed
4.21.19 @1:59pm - (2/ ) (Replying to @yudapearl @ahmaurya) which starts and ends with regressional questions and, surprisingly, invokes a causal diagram. "For what purpose?" I asked. But may I suggest that, instead of putting words in my mouth, please articulate the research question you claimed I have been avoiding. #Bookofwhy

4.21.19 @12:08am - (1/2) (Replying to @ahmaurya) Are you asking why I am surprised to find "regression analysts seeking the wisdom of causal diagrams when they are not asking causal questions"? Ans. Because I have not seen it done in the regression literature, not even in the econometric literature and its "tricks". But
4.21.19 @12:21am - (2/2) (Replying to @yudapearl @ahmaurya) I thought you have earlier accused me of avoiding a burning research question, the answer to which is revealed in the econometric literature. Glad we are no longer there. Or, if we are, what is that question that I avoid? #Bookofwhy

4.19.19 @11:20pm - (Replying to @ahmaurya) I wish I knew what I have done wrong to earn this Tweet. What questions are you asking whose answers I avoid? Try me, I would love to learn from you and the books you read.

4.19.19 @4:54am - (1/3) I have read this paper with great interest, trying to understand what makes regression analysts seek the wisdom of causal diagrams when they are not asking causal questions and labor merely to assess the magnitude of measurement errors. The answer seems to be two fold.
4.19.19 @4:54am - (2/3) (1) The diagram allows them to use Wright's Rules to compute correlations among latent variables (X,Y) in terms of correlations among observed proxies (x',Y'). This could be done, of course, w/o the diagram, but only at the cost of painful algebraic
4.19.19 @4:54am - (3/3) derivations, as in econ. (2) The problem is in fact causal in disguise. Why else would anyone be interested in cov(X,Y) as opposed to cov(X',Y') which is estimable from the data and is sufficient for all predictive tasks? Curious if other readers agree. #Bookofwhy

4.18.19 @2:09pm - (Replying to @y2silence) Amazing discovery!! Unveiling the origin of ideas. #Bookofwhy

4.18.19 @2:06pm - (Replying to @abesilbe) I just spoke with students from NYU Realize-Israel. Harrassment, threats and intimidation is unfortunately the modus operandi of the NYU SJP chapter. Please speak to them. Curious, what gives you the hope that they would be different?

4.18.19 @1:15pm - (Replying to @abesilbe) I fail to grasp the logic of this new theory of evidence. You attended a symposium that was not interrupted by SJP, from which we should infer that SJP does not resort to disruption tactics nation wide, and did not interrupt the May 17 "indigenous people" meeting at UCLA?

4.18.19 @4:33am - An update from NYU: ... It seems that NYU administrators have discovered a new and courageous way of handling disruptive student organizations: Give them awards and do not show up to the ceremony. Dont ask, dont tell.

4.17.19 @4:33am - (Replying to @arnoldroth @epavard) And a joyful Passover to you @arnoldroth, and to your family. We will never forget your beautiful daughter Malki. She and my son Daniel are the torches to the light of which this insane world may discover one day why the normalization of evil is twice as evil as evil itself.

4.17.19 @2:33am - (Replying to @HolgerSteinmetz) Strange, I feel the opposite. My impression is that, at least on this Tweeter forum, the number of such causality-dead people is shrinking by the hour. But this impression is infected of course by heavy (and hopeful) selection bias. #Bookofwhy

4.16.19 @4:19pm - (1/ ) This sad happening at NYU unveils the power of ignorance in the electronic age. I bet my esteem colleagues at NYU do not know that their university is awarding a "president service award" to SJP, a student organization that prides itself on crushing meetings of other student
4.16.19 @4:19pm - (2/2) organizations. How can you tell when your university administrators are embarrassed by their own words? When they start lecturing you on "free speech" -- the ultimate blanket for inaction or lack of courage. Any NYU alumni on this Tweeter?

4.15.19 @3:29am - (Replying to @AngeloDalli) Interesting. But we need to ask Miguel if this is what he meant by "description". His examples do not match this interpretation. #Bookofwhy

4.15.19 @12:39am - In the interest of many readers of Primer who requested to see a derivation of Eqs. 4.13 and 4.14, I am re-tweeting here an earlier post by Ang Li. It may look complicated, but it is a straightforward application of probability calculus. Thanks @Ang_UCLA

4.15.19 @11:51pm - (1/2) I believe it is a mistake to assume that business applications care only about interventions, not about counterfactuals. An astute businessman wishes to spend his advertisement budget on people who are "swayable potential customers" not on "captive buyers". The distinction
4.14.19 @11:51pm - (2/2) between these two groups is counterfactual in nature, and requires counterfactual logic for definition, analysis, identification and estimation. There is more to counterfactuals than meets the eye, and it is all here: , told and exemplified. #Bookofwhy

4.14.19 @3:33pm - (Replying to @DrJohnKang @cd_fuller @julian_hong) Precisely, except that we need to add: "...would behave under conditions different from those Joe's encountered in the the trial." See . Also, the word "generalizability" is reserved for extrapolations across diverse populations. #Bookofwhy.

4.14.19 @2:39pm - (1/ ) An important correction. I actually do NOT agree with Hernan's classification of #datascience. First, I do not see substantive difference between "description" and "prediction'. Second, the "counterfactual" layer should be split into two, intervention and counterfactuals, as
4.14.19 @2:39pm - (2/2) in the Ladder of Causation #Bookofwhy or . The reason is that these two layers of the ladder require different types of knowledge. You can never tell if "Joe's headache would have gone had he not taken aspirin" by conducting RCT on aspirin and headache.

4.14.19 @5:17