{"id":162,"date":"2026-05-09T09:54:00","date_gmt":"2026-05-09T09:54:00","guid":{"rendered":"https:\/\/TheGinoCase.info\/?page_id=162"},"modified":"2026-05-10T08:25:02","modified_gmt":"2026-05-10T08:25:02","slug":"162-2","status":"publish","type":"page","link":"https:\/\/TheGinoCase.info\/?page_id=162","title":{"rendered":"Transcripts"},"content":{"rendered":"\n<!-- ===== The Gino Case \u2014 Transcripts =====\n     Paste this entire block into a WordPress \"Custom HTML\" block on your page.\n     Everything is scoped under the wrapper class .gino-tx so it won't conflict\n     with the WordPress theme. -->\n\n<link rel=\"preconnect\" href=\"https:\/\/fonts.googleapis.com\">\n<link href=\"https:\/\/fonts.googleapis.com\/css2?family=Montserrat:wght@400;500;600;700;800&#038;family=Source+Serif+4:opsz,wght@8..60,400;8..60,500;8..60,600;8..60,700&#038;display=swap\" rel=\"stylesheet\">\n\n<style>\n.gino-tx {\n    --ink: #0e0e10;\n    --ink-soft: #2a2a2e;\n    --ink-mute: #5a5a62;\n    --paper: #ffffff;\n    --paper-warm: #faf8f4;\n    --rule: #e2ddd4;\n    --accent: #1c64f2;\n    --accent-deep: #1d4fd6;\n    --hl-larry: #2b3a55;\n    --hl-francesca: #8a3324;\n    --hl-ron: #2f5d3a;\n    --hl-ava: #800000;\n    --hl-other: #4a3a6b;\n    --serif: \"Source Serif 4\", Georgia, \"Iowan Old Style\", \"Times New Roman\", serif;\n    --sans: \"Montserrat\", system-ui, -apple-system, \"Helvetica Neue\", Arial, sans-serif;\n  }\n  .gino-tx * { box-sizing: border-box; }\n  .gino-tx { scroll-behavior: smooth; scroll-padding-top: 24px; }\n  \n\n  .gino-tx \/* Site chrome (mimics WP theme; remove when embedding on the live site) *\/\n  .site-chrome { background: #0a0a0a; padding: 14px 0 0; }\n  \n  .gino-tx \/* Pushed below content with low opacity so it never overlaps title\/nav *\/\n  .site-header::before {\n    content: \"\";\n    position: absolute;\n    inset: 0;\n    background-image: url(\"data:image\/svg+xml;utf8,<svg xmlns='http:\/\/www.w3.org\/2000\/svg' width='600' height='400'><g fill='%23000' fill-opacity='0.045' font-family='Times New Roman, serif' font-weight='900' font-size='180' letter-spacing='-4'><text x='-30' y='180'>EVIDENCE<\/text><text x='220' y='340' transform='rotate(-3 220 340)'>HEARING<\/text><text x='-10' y='400'>TENURE<\/text><\/g><\/svg>\");\n    background-size: 800px auto;\n    background-repeat: repeat;\n    opacity: 0.6;\n    z-index: 0;\n    pointer-events: none;\n  }\n  \n  \n  \n  \n  \n\n  \n\n  .gino-tx .page { background: var(--paper); padding: 0 24px 120px; }\n  .gino-tx .page-inner { max-width: 880px; margin: 0 auto; }\n  .gino-tx .page-title-block { padding: 40px 0 28px; border-bottom: 1px solid var(--rule); }\n  .gino-tx .page-eyebrow { font-size: 13px; font-weight: 600; letter-spacing: 0.18em; text-transform: uppercase; color: var(--ink-mute); margin-bottom: 18px; }\n  .gino-tx .page-title { font-family: var(--sans); font-weight: 800; font-size: clamp(40px, 5.5vw, 60px); line-height: 1.05; margin: 0 0 18px; letter-spacing: -0.015em; }\n  .gino-tx .page-title em { font-style: normal; color: var(--ink-mute); font-weight: 500; }\n  .gino-tx .page-lede { font-family: var(--serif); font-size: 19px; line-height: 1.6; color: var(--ink-soft); margin: 0; max-width: 64ch; }\n\n  .gino-tx .index-section { padding: 56px 0 24px; }\n  .gino-tx .index-heading { font-family: var(--sans); font-weight: 700; font-size: 13px; letter-spacing: 0.22em; text-transform: uppercase; color: var(--ink-mute); margin: 0 0 22px; display: flex; align-items: center; gap: 14px; }\n  .gino-tx .index-heading::after { content: \"\"; flex: 1; height: 1px; background: var(--rule); }\n  .gino-tx .ep-list { list-style: none; padding: 0; margin: 0; border-top: 1px solid var(--rule); }\n  .gino-tx .ep-card { border-bottom: 1px solid var(--rule); }\n  .gino-tx .ep-card-link { display: grid; grid-template-columns: 88px 1fr 32px; align-items: center; gap: 24px; padding: 22px 8px; text-decoration: none; color: inherit; transition: background 0.15s, padding 0.2s; }\n  .gino-tx .ep-card-link:hover { background: var(--paper-warm); padding-left: 18px; }\n  .gino-tx .ep-card-link:hover .ep-card-arrow { transform: translateY(2px); color: var(--accent); }\n  .gino-tx .ep-card-num { font-family: var(--sans); font-weight: 800; font-size: 38px; color: var(--ink); letter-spacing: -0.02em; line-height: 1; }\n  .gino-tx .ep-card-body { display: flex; flex-direction: column; gap: 4px; min-width: 0; }\n  .gino-tx .ep-card-title { font-family: var(--sans); font-weight: 700; font-size: 21px; color: var(--ink); letter-spacing: -0.01em; }\n  .gino-tx .ep-card-blurb { font-family: var(--serif); font-size: 16px; line-height: 1.5; color: var(--ink-mute); }\n  .gino-tx .ep-card-arrow { font-family: var(--sans); font-size: 22px; color: var(--ink-mute); text-align: right; transition: transform 0.2s, color 0.2s; }\n\n  .gino-tx .episode { padding-top: 96px; margin-top: -1px; }\n  .gino-tx .ep-header { border-top: 3px solid var(--ink); padding-top: 28px; margin-bottom: 36px; }\n  .gino-tx .ep-eyebrow { font-family: var(--sans); font-size: 12px; font-weight: 600; letter-spacing: 0.18em; text-transform: uppercase; color: var(--ink-mute); margin-bottom: 14px; }\n  .gino-tx .ep-title { font-family: var(--sans); font-weight: 800; font-size: clamp(32px, 4.5vw, 44px); line-height: 1.08; margin: 0 0 14px; letter-spacing: -0.015em; }\n  .gino-tx .ep-blurb { font-family: var(--serif); font-style: italic; font-size: 19px; line-height: 1.55; color: var(--ink-soft); margin: 0; max-width: 60ch; }\n  .gino-tx .ep-body { font-family: var(--serif); font-size: 19px; line-height: 1.72; color: var(--ink); }\n  .gino-tx .ep-body .para { margin: 0 0 1.05em; text-wrap: pretty; hyphens: auto; }\n  .gino-tx .ep-body .para.has-dropcap::first-letter { font-family: var(--sans); font-weight: 800; font-size: 4.6em; line-height: 0.86; float: left; padding: 6px 12px 0 0; margin-top: 4px; color: var(--ink); }\n\n  .gino-tx .speaker-tag { font-family: var(--sans); font-size: 11px; font-weight: 700; letter-spacing: 0.22em; text-transform: uppercase; color: var(--hl-larry); margin: 1.6em 0 0.55em; display: inline-flex; align-items: center; gap: 10px; padding: 4px 12px 4px 10px; background: rgba(43, 58, 85, 0.06); border-radius: 999px; }\n  .gino-tx .speaker-tag .speaker-dot { width: 7px; height: 7px; border-radius: 50%; background: currentColor; display: inline-block; }\n  .gino-tx .speaker-tag.sp-francesca { color: var(--hl-francesca); background: rgba(138,51,36,0.07); }\n  .gino-tx .speaker-tag.sp-ron { color: var(--hl-ron); background: rgba(47,93,58,0.08); }\n  .gino-tx .speaker-tag.sp-ava { color: var(--hl-ava); background: rgba(128, 0, 0, 0.08);}\n  .gino-tx .speaker-tag.sp-other { color: var(--hl-other); background: rgba(74,58,107,0.07); }\n  .gino-tx .ep-body > .speaker-tag:first-child { margin-top: 0; }\n\n  .gino-tx .ep-footer { margin-top: 44px; padding-top: 22px; border-top: 1px solid var(--rule); display: flex; justify-content: space-between; gap: 16px; flex-wrap: wrap; }\n  .gino-tx .ep-footer-link { font-family: var(--sans); font-size: 14px; font-weight: 600; color: var(--accent); text-decoration: none; }\n  .gino-tx .ep-footer-link:hover { color: var(--accent-deep); text-decoration: underline; }\n\n  .gino-tx .back-to-top { position: fixed; right: 28px; bottom: 28px; width: 52px; height: 52px; border-radius: 50%; border: none; background: var(--ink); color: #fff; font-family: var(--sans); font-size: 22px; font-weight: 700; cursor: pointer; box-shadow: 0 6px 24px rgba(0,0,0,0.18), 0 2px 6px rgba(0,0,0,0.12); opacity: 0; transform: translateY(12px); pointer-events: none; transition: opacity 0.2s, transform 0.2s, background 0.15s; z-index: 50; }\n  .gino-tx .back-to-top.visible { opacity: 1; transform: translateY(0); pointer-events: auto; }\n  .gino-tx .back-to-top:hover { background: #000; }\n  .gino-tx .back-to-top:focus-visible { outline: 3px solid var(--accent); outline-offset: 3px; }\n  .gino-tx #index { scroll-margin-top: 24px; }\n\n  @media (max-width: 640px) {\n    .gino-tx .ep-card-link { grid-template-columns: 56px 1fr 24px; gap: 16px; padding: 18px 4px; }\n    .gino-tx .ep-card-num { font-size: 28px; }\n    .gino-tx .ep-card-title { font-size: 18px; }\n    .gino-tx .ep-card-blurb { font-size: 15px; }\n    .gino-tx .ep-body { font-size: 18px; line-height: 1.65; }\n    .gino-tx .ep-body .para.has-dropcap::first-letter { font-size: 3.8em; }\n    .gino-tx .back-to-top { width: 46px; height: 46px; right: 16px; bottom: 16px; font-size: 18px; }\n    .gino-tx .site-header { padding: 40px 16px 56px; }\n  }\n  @media print {\n    .gino-tx .back-to-top, .gino-tx .site-chrome, .gino-tx .torn-edge, .gino-tx .ep-footer { display: none; }\n    .gino-tx .page { padding: 0; }\n    .gino-tx .episode { page-break-before: always; padding-top: 0; }\n    .gino-tx .episode:first-of-type { page-break-before: auto; }\n    \n  }\n<\/style>\n\n<div class=\"gino-tx\">\n\n<main class=\"page\">\n  <div class=\"page-inner\">\n\n    <div class=\"page-title-block\" id=\"gino-top\">\n      <div class=\"page-eyebrow\">The Law, Such As It Is &middot; Season 3<\/div>\n      <h1 class=\"page-title\">The Gino Case <em>\u2014 Transcripts<\/em><\/h1>\n      <p class=\"page-lede\">Full transcripts of all nine episodes of the third season of the podcast &#8220;The Law, Such As It Is,&#8221; reviewing the tenure revocation of Harvard Business School Professor Francesca Gino.<\/p>\n    <\/div>\n\n    <section class=\"index-section\" id=\"gino-index\" aria-labelledby=\"gino-index-heading\">\n      <h2 id=\"gino-index-heading\" class=\"index-heading\">Episodes<\/h2>\n      <ol class=\"ep-list\">\n        <li class=\"ep-card\"><a class=\"ep-card-link\" href=\"#gino-episode-1\"><span class=\"ep-card-num\">01<\/span><span class=\"ep-card-body\"><span class=\"ep-card-title\">The Story Begins<\/span><span class=\"ep-card-blurb\">Larry Lessig introduces the season and the essential elements of the Francesca Gino case.<\/span><\/span><span class=\"ep-card-arrow\" aria-hidden=\"true\">\u2193<\/span><\/a><\/li><li class=\"ep-card\"><a class=\"ep-card-link\" href=\"#gino-episode-2\"><span class=\"ep-card-num\">02<\/span><span class=\"ep-card-body\"><span class=\"ep-card-title\">Inside the Investigation<\/span><span class=\"ep-card-blurb\">How Harvard Business School concluded that Francesca had committed academic misconduct, and the gag order that prevented her defense.<\/span><\/span><span class=\"ep-card-arrow\" aria-hidden=\"true\">\u2193<\/span><\/a><\/li><li class=\"ep-card\"><a class=\"ep-card-link\" href=\"#gino-episode-3\"><span class=\"ep-card-num\">03<\/span><span class=\"ep-card-body\"><span class=\"ep-card-title\">The Hearing &amp; Tenure Revocation<\/span><span class=\"ep-card-blurb\">The two-year process that ended in the first revocation of tenure in Harvard&#8217;s history.<\/span><\/span><span class=\"ep-card-arrow\" aria-hidden=\"true\">\u2193<\/span><\/a><\/li><li class=\"ep-card\"><a class=\"ep-card-link\" href=\"#gino-episode-4\"><span class=\"ep-card-num\">04<\/span><span class=\"ep-card-body\"><span class=\"ep-card-title\">Allegation #2<\/span><span class=\"ep-card-blurb\">A close look at the evidence behind the second of four allegations of data fraud.<\/span><\/span><span class=\"ep-card-arrow\" aria-hidden=\"true\">\u2193<\/span><\/a><\/li><li class=\"ep-card\"><a class=\"ep-card-link\" href=\"#gino-episode-5\"><span class=\"ep-card-num\">05<\/span><span class=\"ep-card-body\"><span class=\"ep-card-title\">A Note on Allegation #2<\/span><span class=\"ep-card-blurb\">A short supplement responding to a question about the Thanksgiving 2014 timeline.<\/span><\/span><span class=\"ep-card-arrow\" aria-hidden=\"true\">\u2193<\/span><\/a><\/li><li class=\"ep-card\"><a class=\"ep-card-link\" href=\"#gino-episode-6\"><span class=\"ep-card-num\">06<\/span><span class=\"ep-card-body\"><span class=\"ep-card-title\">Allegation #4<\/span><span class=\"ep-card-blurb\">Examining the evidence behind allegation number four.<\/span><\/span><span class=\"ep-card-arrow\" aria-hidden=\"true\">\u2193<\/span><\/a><\/li><li class=\"ep-card\"><a class=\"ep-card-link\" href=\"#gino-episode-7\"><span class=\"ep-card-num\">07<\/span><span class=\"ep-card-body\"><span class=\"ep-card-title\">Allegation #3<\/span><span class=\"ep-card-blurb\">Examining the evidence behind allegation number three.<\/span><\/span><span class=\"ep-card-arrow\" aria-hidden=\"true\">\u2193<\/span><\/a><\/li><li class=\"ep-card\"><a class=\"ep-card-link\" href=\"#gino-episode-8\"><span class=\"ep-card-num\">08<\/span><span class=\"ep-card-body\"><span class=\"ep-card-title\">Allegation #1<\/span><span class=\"ep-card-blurb\">The only allegation within Harvard&#8217;s six-year limitation rule.<\/span><\/span><span class=\"ep-card-arrow\" aria-hidden=\"true\">\u2193<\/span><\/a><\/li><li class=\"ep-card\"><a class=\"ep-card-link\" href=\"#gino-episode-9\"><span class=\"ep-card-num\">09<\/span><span class=\"ep-card-body\"><span class=\"ep-card-title\">Closing Argument<\/span><span class=\"ep-card-blurb\">A closing argument: drawing together everything across the prior episodes.<\/span><\/span><span class=\"ep-card-arrow\" aria-hidden=\"true\">\u2193<\/span><\/a><\/li>\n      <\/ol>\n    <\/section>\n\n    <section class=\"episode\" id=\"gino-episode-1\" data-screen-label=\"01 The Story Begins\"><header class=\"ep-header\"><div class=\"ep-eyebrow\">Episode 01 &middot; The Law, Such As It Is &middot; Season 3<\/div><h2 class=\"ep-title\">The Story Begins<\/h2><p class=\"ep-blurb\">Larry Lessig introduces the season and the essential elements of the Francesca Gino case.<\/p><\/header><div class=\"ep-body\"><div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para has-dropcap\">This is Larry Lessig. This is the first episode of the fourth [actually, third] season of the podcast, &#8220;The Law, such as it is.&#8221;<\/p>\n<p class=\"para\">If you know this podcast, you know that the seasons are tied to a particular legal case or story, or law like related case or story, and this season is tied to the extraordinarily depressing story of Francesca Gino, who is a Harvard professor in the business school who this year, was terminated by the Business School and the University on charges of academic misconduct.<\/p>\n<p class=\"para\">My relationship to this story at the beginning was as a friend and then, as the story unfolded, I accepted some obligations as a pro bono lawyer, finding other lawyers to support Francesca and finding litigation support, and then litigation funding. And then I finally was involved just in the appeal of the decision of the hearing committee to the Corporation and the President, an appeal which I failed to prevail in.<\/p>\n<p class=\"para\">So I\u2019m not neutral in this story, but I don\u2019t intend to tell this story as a neutral account. I intend to tell it in a way that helps you understand exactly why I believe, as I firmly do, that this story is a story of an innocent person, wrongfully called guilty, and the consequences for her, her family and for, I think, the university are quite profound.<\/p>\n<p class=\"para\">And so this story will, over the course of a number of episodes, unpack this really complicated account of what happened in a way designed to make it so that you, if you are a careful listener or an interested listener, can understand the charges and the evidence and whether, in the end, you believe the charges, given the evidence.<\/p>\n<p class=\"para\">As we go through the story I will post to the website, which is TheGinoCase.info, and also on the Substack, which you can get off of my Substack and link from the website the papers or documents or evidence or argument that\u2019s relevant to that particular moment of the podcast.<\/p>\n<p class=\"para\">The Design. My intent is to tell the story in bite size chunks so that you can follow along, maybe in real time, or maybe you\u2019ll listen to this long after we\u2019re finished, and understand how each part hangs together to reach the conclusion, which I firmly believe, that Francesca Gino was innocent.<\/p>\n<p class=\"para\">In this episode, I\u2019m going to explain what I hope the podcast will accomplish and why. But we\u2019ll begin by introducing the essential elements of the story that got this case going.<\/p>\n<p class=\"para\">In July 2021, Data Colada, which is a website that hosts investigative reports flagging academic empirical work that seems to have problems with the data supporting that work, contacted the Harvard Business School to inform them that it had identified what I\u2019m going to call in this series data anomalies in four papers by Francesca Gino. Now by data anomalies, I mean problems in the data that make it seem as if the data is fake or faked or supplied rather than actually data which should independently support the conclusions of the paper.<\/p>\n<p class=\"para\">Three months later, after Data Colada contacted HBS, HBS informed Francesca of these charges, and then instructed her not to discuss this issue with anyone except two advisors that she was permitted to appoint. So this is the beginning of the gag rule that blocked Francesca from being able to engage in public discussion about these charges and defend herself. And that\u2019s significant, because if you\u2019re coming to the story, you\u2019re coming to it most likely with a lot of information on the other side. There have been articles in major outlets such as the New Yorker and the Atlantic that have recounted the charges against Francesca. But you\u2019ll find, if you pay attention in those articles, there\u2019s basically no defense offered by Francesca to those charges. And that was in part because, well, in the whole, in the main, completely because, as she was going through this process in order to comply with the rules the university and the business school were imposing on her, she was not allowed to provide the information, the defense. She wasn\u2019t allowed to tell her story.<\/p>\n<p class=\"para\">So at this initial stage, I didn\u2019t know anything about this story either, because I wasn\u2019t one of the two advisors that she appointed. These were two people from inside the business school, so I never heard about this until later. But the important thing to think about, if you think about the time between the first notification to the business school, July 2021 and then October 2021, when Francesca was notified of these charges, is that between October 2021 and June 13, 2023, so almost two years, Francesca lived under this gag order.<\/p>\n<p class=\"para\">Now there\u2019s lots that happened during these two years, and we\u2019re going to get to what happened in the later episodes. But on June 13, 2023, the Dean of the Business School, Dean Datar informed Francesca that the investigation had concluded, and that it had concluded that she had committed academic misconduct. And then the Dean, through a colleague, tried to advise her that she should go quietly, and if she does go quietly, then the business school would say nothing more about why, in fact, she would leave. But Francesca refused to go quietly, because, as she insisted then and as she continues to insist now, she had done nothing wrong.<\/p>\n<p class=\"para\">Okay, so after the business school concluded that she was guilty of academic misconduct, it asked the university to begin what\u2019s called a Third Statute Proceeding. Third Statute, meaning: the Third Statute governing Harvard University, and that\u2019s the procedure for revoking someone\u2019s tenure. In the whole of the history of Harvard University, no one had had their tenure revoked under the Third Statute proceeding. Lots of people charged with all sorts of wrongdoing, and those people eventually decided to go away, as she was advised to do by the business school. None of them stayed to fight. Francesca decided to fight because, again, as Francesca believed she was not guilty of anything at all.<\/p>\n<p class=\"para\">So when this university proceeding began, that again, put Gino under a new and different gag order: she was not permitted to discuss the tenure revocation proceeding until it was complete.<\/p>\n<p class=\"para\">And though these proceedings were supposed to be an academic process, where academics get together and try to decide whether they believe the evidence supports the claim that some type of misconduct had occurred, this proceeding turned out to be more lawyers than academics. Francesca was represented by lawyers. HBS was represented by lawyers. The hearing committee that ultimately would decide whether her tenure would be revoked was also counseled by lawyers. And Harvard involved its own General Counsel office in the proceeding as well. So a gaggle of lawyers, all working out the question whether the evidence supported the claim that Francesca had committed academic misconduct with respect to these four articles identified by Data Colada.<\/p>\n<p class=\"para\">Eighteen months later \u2014 eighteen months later \u2014 the Hearing Committee, which was a faculty committee composed of seven faculty members from five departments, including one from the law school, heard evidence. Two months after that, the Committee concluded that Francesca was guilty. Now, as I said, I wrote the appeal to the President and Corporation from that finding, submitting this 25-page document in March of 2025. But in May, the university informed Francesca that it would accept the findings of the Hearing Committee, and effectively immediately she was removed from the University.<\/p>\n<p class=\"para\">Okay, as I said, the aim of this podcast series is to unpack that story. In principle, we could write, I could write a 50,000 word \u2014 I don\u2019t know how many words it would take \u2014 article, essay, book, to tell the story in a way that may clear what I believe. But I think it\u2019s going to be more useful to tell this story in a way that people can listen to and hear. In the exchange that I\u2019m going to have with the \u2014 well, he is, in my view, kind of a \u2014 genius in unpacking the evidence in this story, and also in a couple episodes with Francesca and with some other people who are involved and can provide some perspective and context, we can hear a better understanding of what happened, and help a neutral listener or an interested listener, or even a motivated listener, come to understand more fully what happened. And what was so wrong about what happened.<\/p>\n<p class=\"para\">As I said, the most regular participant I\u2019ll introduce in a later episode, he\u2019s going to help us understand the facts and the statistics. I\u2019ve invited Data Colada to participate in this podcast. That might seem odd. They started this whole thing. Why would I want to include them in the story? But the truth is, I\u2019m a fan of the work of Data Colada, and I believe what they do is incredibly important, and so I think they should be a part of this conversation. And I hope they are.<\/p>\n<p class=\"para\">Depending on how the facts unpack, it might be helpful to include the expert who prepared the analysis that was submitted to the university as well, but Data Colada is the core beginning of this story, and I hope they will, with us, unpack that evidence in light of what was discovered about that evidence in the investigation and hearing. We\u2019ll see. And as I said a couple times during this podcast, you\u2019ll hear from Francesca directly.<\/p>\n<p class=\"para\">So why is this a series that you should listen to? I mean, there\u2019s so much out there that you could be listening to, so much going on, so much that is existential to the nation, depending on your P(Doom) to humanity, to the universe. I guess the answer to that depends upon who you are. If you\u2019ve already heard about this case to any extent, I hope you\u2019ll listen, if only to hear the other side of the story. Lots of people have said lots of stuff about Francesca and about this case, most of it with absolute confidence. It\u2019s kind of hilarious when I read these absolutely certain claims by people who basically knew nothing. But that\u2019s the nature of the internet these days. It brings out the best of the worst of us in hilarious color.<\/p>\n<p class=\"para\">So you\u2019ve heard maybe lots about this case, but you\u2019ve never heard her side. And as I\u2019ve said, that\u2019s because she\u2019s effectively been gagged during almost all of these past four years. So you might believe you know the truth. I\u2019m not going to call you out for that, but just give me some time and listen at whatever pace you want. And after you\u2019ve heard everything, just ask yourself again, were you right, originally? Are you confident you were right?<\/p>\n<p class=\"para\">Because to me, a lawyer, a law professor who teaches at Harvard Law School, that\u2019s the bit that gets me the most about this case. As Francesca\u2019s lawyers presented her case to the hearing committee that would ultimately decide to remove her tenure \u2014 as I reviewed the case that they presented at the stage at which I was engaging in writing the appeal of the decision \u2014 I was pretty confident that that evidence would have led the committee to conclude that they didn\u2019t have the evidence to decide that she had committed academic misconduct. Why? Because the question the committee was supposed to answer was not, &#8220;Is it possible that Francesca committed academic misconduct?&#8221; It was not even, &#8220;Is it more likely than not that Francesca committed academic misconduct?&#8221; The question they were supposed to answer is: &#8220;Was there clear and convincing evidence that she committed academic misconduct?&#8221;<\/p>\n<p class=\"para\">Clear and convincing evidence. Now any lawyer will recognize that\u2019s an extraordinarily high bar. It\u2019s not quite the no reasonable doubt standard that a jury must meet in order to find somebody guilty of a crime, but it plainly requires clear proof of a wrong. Clear proof.<\/p>\n<p class=\"para\">Now, as I\u2019ve said, I\u2019m convinced Francesca did no wrong. I\u2019m convinced she is innocent, absolutely.<\/p>\n<p class=\"para\">But I would concede that there\u2019s at least a conceivable scenario to support the idea she\u2019s not innocent. Conceivable. I\u2019m going to describe that as the \u2018evil genius scenario\u2019 later on that could show how indeed she could be guilty. But I\u2019m quite certain that scenario is just fantasy. And while I don\u2019t think anybody could fairly conclude that it\u2019s more likely than not that she committed academic fraud, I guess that\u2019s a possible conclusion. We\u2019ll hold that idea as we work through the evidence and see whether it can be sustained.<\/p>\n<p class=\"para\">But what I am certain of, beyond being certain that she is innocent, what I am certain of, as a lawyer, is that there is no way a fact finder could fairly conclude that there is clear and convincing evidence of her guilt. To anyone who knows anything about such standard, such a conclusion is just crazy talk.<\/p>\n<p class=\"para\">Okay, which will bring us to the final points I want to make in this first brief episode of this podcast. Crazy talk by Harvard University. Now you might have heard of Harvard University. It\u2019s been in the news a lot recently because President Trump is doing everything in his power to force Harvard to bend the knee to him, and essentially turn over to him and his cronies, the management of this, the oldest university in America.<\/p>\n<p class=\"para\">This won\u2019t be a podcast about that outrage. Suffice it here that I believe the actions of the President are illegal and unconstitutional, and so will the courts eventually conclude. But many institutions and individuals have caved to the President\u2019s threats, technically extortionate threats. Harvard hasn\u2019t. Harvard is fighting the illegal threats and punishments being thrown against it by a President with an aggressive and effective legal defense. I have enormous respect for Harvard doing this. Actually, I have enormous respect for the President of Harvard, Alan Garber, and the Corporation, in choosing to do the right thing rather than the easy, or at least easier thing.<\/p>\n<p class=\"para\">No one should doubt that Harvard is going to suffer dearly, even when we win. That suffering is a true act of integrity. And I would say even more. I\u2019ve taught at many law schools, Chicago, Yale, Stanford. This is my second gig at Harvard. I\u2019ve been here this time since 2009 and I love it here. I love the Law School. I have enormous respect for my colleagues. They are among my closest friends, and I have never known a more talented and inspiring student body. It couldn\u2019t be better for a person with a job like mine.<\/p>\n<p class=\"para\">But one can love and respect a person or an institution, and yet believe they have committed a mistake, indeed a great wrong. That is what I believe here. And this wrong is not just an abstract wrong, it\u2019s a wrong that has effectively ended the academic career of an extraordinary teacher and scholar. At the very least, we owe her the best statement of her defense, so that anyone keen to understand at least this version of the truth, which turns out to be the right version of the truth, can.<\/p>\n<p class=\"para\">So in telling somebody I was going to do this, they said to me, aren\u2019t you afraid of what Harvard will do to you if you do this? And when this friend said this to me, I realized both how extraordinary this moment is in American history and how much faith I still have at this moment in American history.<\/p>\n<p class=\"para\">If I were taking on President Trump in a way that would matter to him, in a way that he would even notice, because, of course, nothing would matter to him that I could do and not that he would ever notice it. But the truth is, if I were taking on President Trump in a way that would matter and he would notice, I would be afraid. The truth is, I\u2019m not sure I would have Harvard\u2019s courage. I\u2019m an oldish guy. I don\u2019t really have another career I could take up. I have three kids still not finished with college. One not even finished with high school. I live in an extraordinarily expensive part of the world. I could not afford not to work. Given everything I\u2019ve seen, if I were taking on Trump, I would be terrified. And though I\u2019d like to tell myself, I would risk it all to do the right thing, like Harvard has, I can\u2019t honestly say that I would.<\/p>\n<p class=\"para\">But at the same time, when my friend asks me this question, am I afraid of Harvard? I realize the enormous faith, maybe just hope I have, about Harvard. Do I believe Harvard would retaliate against me for stating publicly what I believe is true? I don\u2019t. I think there is zero chance that Harvard would punish me for criticizing it or them, at least, so long as the story stays true and appropriately respectful. I could never imagine them retaliating. That is what I thought when my friend asked me that question. That is what I believe, because I believe that is what America and Harvard and every great institution should be.<\/p>\n<p class=\"para\">But then, as I thought that, I realized that six months ago, I could never have imagined where we are today, with Trump illegally threatening Harvard, news organizations, law firms, governors, essentially any institution independent of him, and so far, at least getting away with it. So I realize my confidence is just a prediction, and I see that it may be totally naive, but I\u2019m going to act on the assumption that the institution whose motto is Veritas, truth, will allow me to describe what I believe is true, even if, maybe especially because, it criticizes Harvard.<\/p>\n<p class=\"para\">So stay tuned for the next episode, which will be one of the few to include Francesca. Thank you for listening.<\/p>\n<p class=\"para\">This podcast is produced by me, not by Equal Citizens. Actually, the production part is done by Josh Elstro of Elstro Productions, but this podcast is independent of the institution of Equal Citizens, and obviously the institution of Harvard. I\u2019m in my basement with my microphone, reading from my notes, looking forward to the next episode.<\/p>\n<p class=\"para\">And one final note about timing, I can\u2019t promise the regular production of these episodes. This story is hard. I discovered how hard it was when I wrote the appeal. I discovered how difficult it was to unpack the story in a way that will make it understandable. And as we will include conversations with others and scheduling those will take time. This season will take time. So if this is a year from now, then certainly you\u2019ll be able to listen to all episodes, which I expect will be between six and eight by the end. But if you\u2019re waiting right now for the next, all you can do is sign up and we will notify you when the next episode drops. Thanks for your patience. Stay tuned for the next episode.<\/p>\n<\/div><footer class=\"ep-footer\"><a class=\"ep-footer-link\" href=\"#gino-index\">\u2191 Back to index<\/a><a class=\"ep-footer-link next\" href=\"#gino-episode-2\">Next: Inside the Investigation \u2192<\/a><\/footer><\/section>\n<section class=\"episode\" id=\"gino-episode-2\" data-screen-label=\"02 Inside the Investigation\"><header class=\"ep-header\"><div class=\"ep-eyebrow\">Episode 02 &middot; The Law, Such As It Is &middot; Season 3<\/div><h2 class=\"ep-title\">Inside the Investigation<\/h2><p class=\"ep-blurb\">How Harvard Business School concluded that Francesca had committed academic misconduct, and the gag order that prevented her defense.<\/p><\/header><div class=\"ep-body\"><div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para has-dropcap\">This is Larry Lessig. This is the second episode of the podcast &#8220;The Law, such as it is.&#8221; We\u2019re in the fourth [actually, third] season where we\u2019re considering the story of Francesca Gino.<\/p>\n<p class=\"para\">Francesca was a professor at the Harvard Business School, and this year, became the first professor in the history of Harvard University to have her tenure removed. The conditions of that removal are the subject of this season of the podcast.<\/p>\n<p class=\"para\">The second episode will cover the events that led to the conclusion by the Harvard Business School that she had committed academic misconduct, and the business school would then recommend that she have her tenure revoked.<\/p>\n<p class=\"para\">As I said in the first episode. I\u2019m not a neutral in this case. I was a friend of, I am a friend of Francesca Gino, and I helped her throughout the process, not as a lawyer, except at the very end, where I helped put together her final appeal to the President, which ultimately failed.<\/p>\n<p class=\"para\">But the purpose of telling this story slowly through the medium of a podcast is to give you a chance to come to understand it carefully, because in the culture of tweeting, it\u2019s an easy story to mischaracterize. In the culture of tweeting, it\u2019s an easy story to mock. I\u2019ve been astonished with the confidence that people who know nothing about the facts seem to have about the facts. So I\u2019m going to try to give you a picture of the facts, and with that picture, you can draw your own conclusions. So this interview happened on August 14, with Francesca in person. We will have one more episode with her telling the balance of the story until she was removed as a professor. And then, as I promised, we will go into four episodes that will each consider the four papers that were said to have been fraudulently produced, and then we\u2019ll have a concluding episode. Stay tuned.<\/p>\n<p class=\"para\">Francesca, I\u2019m grateful that you\u2019re having this conversation with me. We\u2019ve known each other for a long time. Why don\u2019t you describe a little bit about when we first got to know each other?<\/p>\n<p class=\"para\">I believe it was around 2010. I had just arrived on campus or soon after. And I knew of you because you were involved in some of the work at the Safra Center, and, at the time, I was doing quite a bit of research on trying to understand why people do what they do, but in the context of morality. And so I came to meet you.<\/p>\n<p class=\"para\">Yes, and then you were affiliated with the Center, and we got to participate in that project, which, at the time, we were focused on what I called &#8220;institutional corruption.&#8221; But that was the focus, really about a social morality, or the morality of systems or institutions. And I just remember you were a vibrant contributor to that conversation, and I was grateful that you participated back then.<\/p>\n<p class=\"para\">But we\u2019re having this conversation today, it\u2019s August 14 2025, because four years ago, in July 2021, a group called Data Colada contacted Harvard Business School, where you were professor, with concerns over four studies in papers you had co-authored. Let\u2019s start with a question of who Data Colada is.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">Data Colada is a group of three behavioral scientists. They do work like other behavioral scientists, trying to understand why people behave the way they do. And at around 2013, they started a blog called Data Colada. They write about other people\u2019s work, they criticize it in the hope of making science better.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">They were, and in some ways, still are, to me, a kind of hero, in the sense of they\u2019re out there trying to make sure that the standards of the field are being met. And, with respect to your work, they claim that there were at least anomalies in the data supporting the claims in your research. Those anomalies could either have been errors or fraud, and they were quite explicit that they didn\u2019t know which. They just knew that there were problems with the data, and so they took their issue to Harvard, which was weird, right? Because they typically would take the issue to the authors. So, did they bring these charges to you at all?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">They did not. It is their policy to go to authors and give them the chance to respond, but they didn\u2019t do that in my case.<\/p>\n<p class=\"para\">Okay, all right. So let\u2019s back up a little bit so everybody understands a little bit more about you and why you do the work you do. So tell me a little bit about how you got to where you are. Obviously from your beautiful accent, it\u2019s clear you didn\u2019t get there from Cambridge, Massachusetts. So how did you get to this position of being a professor at the Harvard Business School?<\/p>\n<p class=\"para\">It\u2019s a bit of a long story. In Italy, when you are in college, you work on a dissertation, and, at the time, I decided to work on one that was about production systems based on a really cool class that I took back then, and I ended up doing a lot of research with the professor who taught the class and wrote a handbook of sort. And I really loved that process. There was also a center, a lab, where I used to go as a participant to be part of lab studies. And so, after graduating from college, I started a PhD program in Italy that was experimental, and the idea is that you would leave on year three and go somewhere else to do your dissertation work. And for no particular good reason, I left at the beginning of my second year. I came to Harvard. And the idea was to say for six months, but I never left. So I started taking classes here. At some point, a professor asked me, &#8220;why not starting a PhD from scratch from here?&#8221; I didn\u2019t understand the system. And so I stayed where I was.<\/p>\n<p class=\"para\">So fast forward, I graduated from my three-year program. I think that everybody understood that I wasn\u2019t quite ready. And so I stayed on as a post doc, followed by another post doc at Carnegie Mellon University. And then I had my first job at UNC, and after a few months, I realized that there wasn\u2019t a department who was doing a lot of behavioral research, and, as an Italian, it was a little difficult to live in Chapel Hill. And so I told myself that it\u2019d be better to move. That was 2009. I received an early offer from Stern. In talking to advisors, they recommended going on the market and doing so selectively in places, cities and universities where I would be happy as a scholar, but also as an Italian.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So that led you here, back to Harvard.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">That\u2019s exactly right.<\/p>\n<p class=\"para\">Okay, but then help us understand a little bit more the nature of the research. So you called yourself a behavioral scientist, but what does it mean to be a behavioral scientist? What does the particular part of the field that you\u2019re working in look like?<\/p>\n<p class=\"para\">Very broadly, Behavioral Science uses psychology as well as economics to try to understand human behavior. It\u2019s really asking questions related to, why do we do the things that we do, or why do we think the way we think? And then you can apply those questions across different settings. And so for some of my research, for example, I studied why is it that good people do bad things? And as you explore these questions, one common way of doing so is through lab experiments or surveys, where you\u2019re trying to put participants, subjects, people like us, in different conditions, and then seeing if, because of that situation or condition, you see a difference in their behavior that is consistent with your hypothesis. So lab studies often start from observations in the world, and then you turn them into hypotheses that you want to test.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so you have a theory, an observation, and you then think about how to test that, and then you might design an experiment that is conducted inside of a lab. And by a lab, we basically mean a room with a bunch of computer terminals, where people come and they sit at the computer terminal and they answer questions. Would that be the way it might look?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">Yeah. And in fact, if you go back to my time at Carnegie Mellon University, or even at UNC, often experiments were in the form of paper surveys. And so you would go around town and ask people if they wanted to participate, and then they would answer by filling out paper surveys, and some of them were condition A and some of them were condition B, and then you would look at the difference that way.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay. Now, when you started doing this, obviously, as a post doc, you did most of the work or all of the work yourself, but as you became an assistant professor, and then a professor here, the character of the team changes, right? Because there\u2019s a ton of data here. There\u2019s a ton of stuff to work through. Who does that ton of work?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">So one of the things that you want to do when you run studies is ensuring that the data is collected properly, and that you are not somehow biasing it in some way, and so you rely on the assistance of what we call the &#8220;research assistants.&#8221; So these are often undergraduates, or people who are taking gap years, or people who want to go to grad school later, and they want to get more research experience, and they are the people who are responsible for collecting the data, cleaning the data. Even before collecting the data, you need approval from the IRB.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">What\u2019s an IRB?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">It\u2019s the Institutional Review Board. They are people who look over your research, your plans for what you intend to do, to make sure that there is nothing that is dangerous or risky for participants. And so the RA would help you write the application to make sure that they have all the information that they need to run the experiment. And it makes sense, then, for them to be the one cleaning the data, since sometimes when you run studies in the lab, not everybody is behaving particularly well, or you might have reasons to drop them, and the RAs have that knowledge.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so the word \u2018cleaning the data\u2019 will strike people who are not in the field as kind of weird. That sounds like you\u2019re, you know, picking the observations you like and ignoring the observations you don\u2019t like. So just give us a practical sense of like, what does messy data look like, and what is done to it to turn it into clean data,<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">Absolutely. So let\u2019s imagine the survey or the experiment asks a question about your age, and often the way the question is asked is, in which year were you born? But some participants going fast, rather than saying 1978, they put 47. And so an RA cleaning the data would correct that, so that when you\u2019re doing descriptive statistics, you\u2019re actually looking at the right data. Or sometimes there is misspelling if you\u2019re giving answers. And so that also needs to be corrected. But cleaning the data also means sometimes you use a survey for testing, and so you have what we call the pilot study. So we invite 10 participants in, and so those need to be removed from the data. And it\u2019s understood from the beginning that they\u2019re not going to be real participants, but an RA would know that, and not the person who\u2019s known as the principal investigator.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so the whole process of becoming an academic in the field that you work is a process of actually learning this incredible, really industry, of producing the data that will then be analyzed to test the hypothesis that people have about why people behave the way they do. So you must have, as a graduate student, been advised by your more senior graduate students or by the professor you are working with, exactly how all this work has done. Done is that is that kind of the culture of how you become a business school professor studying behavioral science.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">Absolutely, it\u2019s part of the training that you receive. And in fact, when I went to Carnegie Mellon University as a post doc, I was actually a visiting assistant professor, to be precise, my position was created for me. And the reason is that they needed people to teach courses to undergraduate around organizational behavior. And I thought I could do that, but they also had a lab manager. So this is a person who oversees all the lab studies that are going on in the lab. And so I took the position of lab manager, I did the teaching, and I became visiting assistant professor. And so my role, day in and out, was to be in the lab, to the be the person, either myself or with the help of other undergraduates, who collected the data for various professors who were working in this field at the time.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So across the United States, how many people are there like you, in the sense of nobody like you, exactly Francesca, but I mean, like you, in the sense of working in behavioral studies like this.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">It\u2019s an interesting question, because some of these scholars sit in business schools. Some of them sit in psychology departments. I would say, when you look at the society where these members tend to aggregate, there are, I believe, 1500 members. It\u2019s the Society of Judgment and Decision Making. And then if you add the psychologists, then I think the number gets a little larger.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so couple 1000 people working in this field. Now, you know, I\u2019ve not been your lawyer in the context of defending you directly, although I did participate at the last stage in writing the final appeal in this process. But I\u2019ll just be frank. When understanding how data is handled in your field, it was a little bit shocking in the sense that it\u2019s as if it doesn\u2019t really matter to be 100% certain that every step has been taken with perfect certainty, because it seems like the process, not just your process, but the process of others, is relatively casual about how data is manipulated, cleaned, passed from one device to another device, then used in a study. Obviously, everybody believing that it\u2019s actually what it\u2019s supposed to be. But if this were a bank, and that\u2019s the way data about, you know, financial transactions were handled, it would be kind of a scandal, right? So am I being fair in my characterizing this as not quite the level of security of data in a bank, and what justifies that difference?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">If I think about this with the head, the mentality and thinking of 2025, I think I\u2019m as surprised as you are. And I look back and say, &#8220;Wait a second, is this how we handled our data?&#8221; And I\u2019m as surprised as you are, even shocked. But when I think about it in the context of the practices of the field in the 2000 and I go back to what we were actually doing, like running around the city trying to gather data from participants, it didn\u2019t seem as strange. Again, it wasn\u2019t just me or my lab. It was everybody in the field having the same type of practices and not exactly thinking through, &#8220;Well, if we collect data on paper, what kind of security do we need around it? Or if we insert data in Excel, what kind of errors are likely to happen?&#8221;<\/p>\n<p class=\"para\">When I think about all the different classes that I took as a PhD student, and I believe this to be the case also in 2025, there are no data management classes. That\u2019s shocking now, reflecting back to what happened to me. But you just learn by looking at what others are doing. And I can\u2019t remember conversations where my colleagues and I look at each other and said, &#8220;Hey, wait a second. Is it possible that&#8230;?&#8221; It just didn\u2019t happen.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Yeah, I mean, one way to understand it is kind of an expression of humility. I mean, you know, really, why does it matter? It\u2019s just an academic paper being published about whether morality is triggered by eggs or by, you know, dish soap. And so if you\u2019re not using the level of security you would use if you were working inside of the Fed, it\u2019s maybe kind of appropriate.<\/p>\n<p class=\"para\">But what\u2019s weird about the moment we\u2019re in is that because of, I think, the legitimate activity of people like Data Colada, there\u2019s a real question that gets raised about, why are there certain anomalies? And you and I both know that the academy is not filled with perfectly pure souls, and there have been very important examples, even especially at Harvard, of people who have overtly manipulated their data. Who have decided I need an answer to get this grant, or I need an answer to get tenure, I need an answer to get some bonus, and so I\u2019ll just fudge the data to get that answer. And to the extent we know that happens, I guess it\u2019s not surprising that we have this conflict between what we could call lax data practices and suspicion triggered by sometimes intentional manipulation, sometimes accidental, unintentional flaws.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">Yeah, so if I go back to 2013 when Data Colada started their blogs, and then it took a few years, as change usually requires, there have been a lot of what I believe are very healthy discussions about looking at practices and ensuring that the conclusions that we draw from the research that we do in behavioral science are solid.<\/p>\n<p class=\"para\">So another one that I think is going to shock you, if you pick up a paper from 2010 or earlier, the number of people that you have in each condition, that you\u2019re using to draw conclusions about your research is probably 20 or 30. And looking at it now, we know that if you do proper power analysis, that is a big mistake. It\u2019s not proper, but it wasn\u2019t known at the time. And so if you\u2019re working in a system where you\u2019re adopting the current practices, and you\u2019re not pausing to ask questions, &#8220;are these the best practices that we have?,&#8221; you end up probably with some anomalies.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">In fact, in this context, we\u2019re going to spend a lot of time not you and I, because you and I are not going to talk about the substance of the papers and whether, in fact, the charges against you are true. I don\u2019t want to put that burden on you, so that\u2019s the conversation I\u2019m going to have with others. But in the context of this, when the charges were made against you, your co-authors were rightly anxious, and they worried, was their work infected by this as well.<\/p>\n<p class=\"para\">And so there was a project, I think it was called the Many Co-authors Project. Describe the Many Co-authors project.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">So, as you said, when the news broke in the summer of 2023, co-authors reacted by saying, &#8220;we should have a database of all the papers that we wrote with Francesca and try to see if there are issues, as far as we know, in the other papers.&#8221; And this was an important effort, because the news rocked the field. There were students who were hitting the market and they needed jobs, and they had my name on their papers or on their CVs, or people going through promotion processes. And so this effort came from the desire to ensure that the papers that others wrote with me were &#8220;clean.&#8221;<\/p>\n<p class=\"para\">And through this effort, one co-author, Juliana Schroeder at the University of Berkeley, decided to audit all the papers she co-authored with me, and she had seven. And as it turns out, in her audits, she ended up finding errors in studies where I didn\u2019t have anything to do with the data collection, data cleaning or data analysis. And to me, that was really an important moment. As I told her, I wish she audited the papers she didn\u2019t co-author with me to try to see how likely errors tend to happen, especially because in some cases, some of the errors were similar to the ones that I uncovered. So, for example, in one of her studies, she found that 34 rows of data were coming from a person taking the survey multiple times, which is something similar to an anomaly that I found in one of my studies. And so, it was just the realization that, unfortunately, errors are common in our field. And to me, it was an eye opener and a call to the field, not only to have better practices, but also better ways to check on the data that we collect.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Now, for many academics, it\u2019s hard to understand what that could mean. So I\u2019m a law professor. I write articles, I do my own research in the sense that I read, and I highlight sections that I\u2019m reading, and I produce endless note cards, and I sit there with the note cards, and I put them to knit them together into an article, and I type out the article, and because I\u2019m a law professor, I obsessively footnote everything. If you discovered in my work, pages of the work that had not been cited that were just, you know, verbatim copies of something else, I think it\u2019s a fair conclusion I did that intentionally. Right? Who else would have done it? And it\u2019s not so hard to just put a footnote, and to the extent I don\u2019t put a footnote, or I repeatedly have stuff that\u2019s not mine in there, I think it\u2019s perfectly fair to assume there must be something wrong with me and my execution in a very intentional way.<\/p>\n<p class=\"para\">Yeah, and even Data Colada, just before the final decision in your case was revealed, turned out to have problems with their data. And I remember reading it, and one of the principals said, you know, it just turns out, this is really hard. And that\u2019s a really important insight: It\u2019s really hard. I don\u2019t think Data Colada fudged their data. I don\u2019t think there was any intentionality to it as well at all, but I think it brings out the fact that this is a field where it\u2019s quite likely that even the most careful are going to turn out to discover that there are problems in the data that they\u2019re working with.<\/p>\n<p class=\"para\">But I think the point I\u2019m trying to emphasize for people who\u2019ve not thought about this endlessly as we have, is that that\u2019s not actually a fair inference to draw in many fields. In order to draw that inference, you got to do more than that and then identify that there\u2019s an anomaly, you\u2019ve got to actually kind of figure out, work out, work backwards, to see where the anomaly comes from, and if it turns out the anomaly comes at the very last stage, when you\u2019re sitting there with the article, trying to put it together, to make it say what you want it to say, Okay, fine: I\u2019m totally willing to believe that the author is responsible there. But if you haven\u2019t even looked at the stages that happened before, the people who were involved in that process, then I think it\u2019s just wrong to leap to the conclusion that this is like, this is criminal behavior. But it\u2019s so hard to resist that. The Atlantic had a piece. It wasn\u2019t Juliana that the piece was about, right?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">Alison Wood Brooks<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Yeah. And she, like through this piece, was trying to help the author understand what was going on and how they could be finding anomalies. And at the end, it turned out she too had anomalies in her paper. But what was striking to me is the author of the piece. The Atlantic pivoted it around right away to say, &#8220;Oh, you too, are a fraudster.&#8221; No, she wasn\u2019t a fraudster. It\u2019s just a messy, difficult, complicated field, and it turns out there\u2019s something wrong with it. That\u2019s not to say there couldn\u2019t be fraud. It\u2019s to say you\u2019ve got to do the work to demonstrate that there\u2019s fraud. You can\u2019t make this leap to the conclusion we\u2019ve got a criminal here, or let\u2019s destroy somebody\u2019s career, because we must have a criminal here, because who else could have done it?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">Yeah, and the piece was about Juliana Schroeder. I\u2019m sad that the reporter concluded or wrote the story the way he did. I think Juliana is a remarkable example of a person who took the moment to say, &#8220;Okay, let\u2019s audit these papers.&#8221; And as she told me, that summer was really hard due to the discovery of the anomalies and errors that she found. But what was also interesting of her story is that what follows were corrections if the anomalies didn\u2019t change the results, or retractions when that was appropriate.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so the bottom line is, you\u2019re in a field where you produce research driven by tons of empirical data. There are many people working on preparing that data for you to analyze. Once the data is prepared, you determine whether it actually shows something interesting, and if it does, you write it up, and if editors at journals like it, they publish it. That\u2019s the state of your life. So how many papers like this have you published?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">I have published about 140 papers. And, generally, papers have multiple studies, since you\u2019re going after similar or a series of hypotheses on the same topic. And so it\u2019s about over 500 studies.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">500 studies that get revealed in 140 papers. And you were charged here with four papers where these anomalies existed, so less than 1% of these papers. I\u2019m a lawyer, so I am doing the math.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">You\u2019re doing the math fast. I\u2019m trying to follow you.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And so in these 140 papers, how many co-authors would be in that mix?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">About 120. Again, this might sound shocking to you if you think about the way you described a paper of yours, but it truly is a different field and a different set of efforts. Again, many papers have five, six, sometimes even seven, studies. And in order to get to those seven studies, you probably conducted many more, because in the first time you try out the experiment, maybe your manipulation wasn\u2019t appropriate in the sense of really proving what you were trying to prove. And so there is a lot of experimentation that goes into the making of these papers, and many people trying to help as the data is being collected to the point where the paper is written up.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay. And those are just the authors. If you had to ballpark, how many people touched the data from the 140 papers that you\u2019ve published, how many people would that be? I mean, obviously a research assistant could work on four or five papers, so it\u2019s not like it would multiply out from 140 but if you just had to, you know, if you think about how many researchers have I worked with? How many lab managers have I worked with? How many post docs have I worked with? Is it like 10s or 20s or hundreds? What does it look like?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">It\u2019s probably in the hundreds. I had at some point during the investigation to look back to all the research associates that worked with me, and I had 66 research assistants who were working for me paid. But then there were many more, actually, that were doing work related to my research or the research I was doing with my graduate students that were doing it for class credit. And so we\u2019re talking about hundreds of people.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, all right, great. So let\u2019s go back to the what happened here in July 2021. Again, four years ago, Data Colada contacts Harvard, not you. Were you told that they contacted Harvard?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">I was not told.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so in July, they contact Harvard. What we know from the discovery in the related proceedings around this issue is that at that point, the business school and Data Colada enter into some sort of agreement. I mean, obviously Data Colada comes to the Business School and says, &#8220;We think you have this fraudster professor, and we\u2019re going to publish these blog posts about her,&#8221; and Harvard\u2019s eager, apparently, to forestall that. And so they enter into an into an agreement where the business school promises to open an investigation, in exchange for them delaying the publication of the blog post. Is that right?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">That\u2019s right.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay. And so they begin that process, but they begin it by creating the process, or more precisely, by rewriting the process that would govern charges of academic misconduct. The Business School, like the law school, had a policy for research misconduct. I think it was a very short kind of two-page policy that had been debated by the faculty and adopted by the faculty as the faculty\u2019s statement of the policy to govern research misconduct.<\/p>\n<p class=\"para\">But, over the summer, the business school crafted a brand new policy: 16 pages that was designed to govern the adjudication of your case. And, again, when they created this, did the faculty debate it and vote on it?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">No, it was done behind closed doors.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so behind closed doors, they change the policy that the faculty had adopted, creating one that\u2019s now eight times longer. The faculty doesn\u2019t know about it. And as you think about the comparison between that policy and the policy which preceded it, what are the two most significant things this policy enforced that the older policy would not necessarily have enforced?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">The two would be, first, the fact that they put a restriction such that I could only engage with and talk to two advisors. And the second important difference is that there were issues of confidentiality, so I could not talk beyond these two advisors, to anyone about what was happening. And the policy states that if I were to share information about the process or what was going on, there would be sanctions up to termination.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so those are two really striking conditions. When I thought about it, it kind of made sense, if you thought of this policy as growing out of the very elaborate process which the universities have adopted to deal with claims of sexual misconduct, where, obviously, confidentiality is really important if you\u2019re going to give victims the space and the permission, the freedom to raise their claims. You must have confidentiality. And if I were managing such a process, I would absolutely say you must maintain confidentiality, and if you breach it, that\u2019s the end. And then the same thing with advisors. If you\u2019re thinking about a rape, or one of these horrendous sexual misconduct claims, as horrendous as it is, it\u2019s within the ken of understanding of ordinary people. We know what we\u2019re talking about. So how many people do you actually need?<\/p>\n<p class=\"para\">So I understand the kind of source of a policy like that, but in this context, it\u2019s quite consequential because this effective gag order means that you know, once you learn of it, so far, you haven\u2019t learned of it&#8230; we\u2019re still talking about them crafting the policy, but once you learn of it, you can\u2019t talk to anybody about it. You can\u2019t talk to the people who you did the research with about it. You can\u2019t talk to your research assistants. You can\u2019t try to figure out what happened, who did what, who said what.<\/p>\n<p class=\"para\">And quite importantly, who knows how many advisors you would need to be able to defend yourself, depending on what the issue was. Because obviously, with your field, unlike mine, they\u2019re really complicated statistical questions to be able to establish whether there\u2019s likelihood of misconduct or not, and that\u2019s beyond my can and at some stage it\u2019s beyond all of our cans. And so that\u2019s why you bring experts in.<\/p>\n<p class=\"para\">Okay, so these policies were enacted, I\u2019m going to call it unfairly maybe, but I\u2019m going to call it the gag order and the restriction on your advisors. And then it also created this new position, the research integrity officer. What is the research integrity officer?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">So you can think of him or her as a person who stands between you and the committee. So the committee is faculty members at Harvard Business School. At the time of the inquiry, it was two, and at the time of the investigation, it was three of them. But it\u2019s not that I could, if I had a question, pick up the phone or write an email directly to the committee. I would always have to do that through the research integrity officer. And similarly, when the committee had questions for me, they would come through this middle person.<\/p>\n<p class=\"para\">Okay, so this is not a faculty person, this is just a staff person who was to be the kind of point of contact between the two. And even though there would be a faculty committee that would investigate and a faculty committee that would then make a determination of whether they thought there was a violation here, you would communicate solely through this research integrity officer.<\/p>\n<p class=\"para\">So, back to the timeline, HBS gets notified in July, they strike a deal with Data Colada. Then for three months, they build this new process for adjudicating charges like this, they hire somebody to manage it. All of this is done in secret. The faculty knows nothing about it. You don\u2019t know anything about it. And then fast forward to October 27 2021. What happens then?<\/p>\n<p class=\"para\">So it is 8.10 in the morning, and I receive an email that I printed out so I have in front of me that said, &#8220;Dear Francesca, I have a serious and time sensitive matter I need to discuss with you today. Could we please meet for 20-30 minutes at your earliest convenience? I understand you may have teaching obligations this morning. Let\u2019s meet on campus if possible, and please bring your HBS issued devices to the meeting. Thanks. Alain.&#8221;<\/p>\n<p class=\"para\">So this is an email from the research integrity officer that I received that morning. And, as it turns out, it was my husband\u2019s 50th birthday. And so I responded by saying, I\u2019m teaching. I have Parent Teachers conferences, and then I have&#8230;<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">for your own kids, parent teachers conference&#8230;<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">yeah, for my own kids. And then I have this big celebration for my husband, since it is his 50th birthday. And he asked me to call him right away before teaching. So I did. And so the day went sideways, where: I taught, and then I walked to campus with my HBS issued devices.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Yeah, I mean, I\u2019m not sure I would have understood what that was about until the &#8220;and bring your HBS issued devices along with you,&#8221; which is obviously a chilling demand.<\/p>\n<p class=\"para\">But I mean, when you think back, you know that we\u2019re talking about July, so August, September, October, three months between the time that they learn of charges of academic misconduct, and you learning about the charges, is that ordinary? Is that like a normal process for academic misconduct?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">No, in fact, I think that the norm is to be told within a week.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Within a week.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">Yeah. And here is over three months.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay. And so you have the conversation, and you learn that you\u2019re being charged with academic misconduct. I can\u2019t imagine how horrible that must be, but when you think back about it, it might have been complicated, because you might have also thought there\u2019s no there there, so I\u2019m not really worried about this. So what did you feel at that moment?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">I remember being shocked. I couldn\u2019t quite understand what I was hearing from the research integrity officer. But then, as you said, I thought maybe there is an error, since I didn\u2019t commit any misconduct. And so, right from the start, I was collaborative, and I did what I was told. I showed up with my HBS issued machines.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so the process begins, you obviously are interacting with the research integrity officer. Describe the research integrity officer, what kind of person?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">So I knew of him, since he used to be one of the people responsible to review applications sent to the IRB. And so if there were ever issues or need for clarification, it would be the person sending comments and having the back and forth.<\/p>\n<p class=\"para\">As it turns out, this position of research integrity officer is really critical. They are in charge of gathering all the data, all the evidence that is relevant in order to understand the allegations against you.<\/p>\n<p class=\"para\">And they manage the process. Again, they\u2019re the person who is communicating with the committee and ensuring that the process is actually followed as is spelled out.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">But was he pleasant? Was he I mean, what kind of relationship did you have on day one with this person?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">So when he asked me to ensure that I would show up at HBS right away, the scene was puzzling, since the police was there. Security was there when I showed up on campus, and so it\u2019s a little bit like being in a movie and sort of asking yourself if this is your story.<\/p>\n<p class=\"para\">I am not very technologically savvy, so not only I showed up with my machines, but I also showed up with my husband, who\u2019s a software engineer, so that I could ensure that if they had questions about my computers, I had an expert on the site to answer them. But it was formal, and I just followed the rules that he set from the start.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so, but he tells you, what? Does he give substance about what\u2019s been charged? Or he says, you\u2019ve been charged with academic misconduct, and this will begin a process of investigation.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">So during the call, when we connected, he let me know that there were allegations of misconduct against me, and that he would send a formal note that would start the inquiry process. And then he told me that for any question that I had, once I read the letter, I could turn to him.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so in this process, you needed to pick two advisors, and only two advisors. Who did you pick and why?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">So the first one I picked was a lawyer, and it wasn\u2019t my choice, really. But it was a suggestion, a very strong suggestion, from the research integrity officer. And so if you go back to that time, again, imagine me, saying, &#8220;What is this all about? Why do I need a lawyer to get involved?&#8221; And so I told him that I didn\u2019t think I needed one.<\/p>\n<p class=\"para\">And he mentioned that in similar investigations in other parts of Harvard, lawyers are common, that he would recommend names of people that I should be talking to. And so one of the advisors ended up being a lawyer. And then, as a second advisor, I chose a mentor and colleague who is a professor at Harvard Business School. His name is Gary Pisano.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so Gary\u2019s a great, obviously superb professor. Is he an expert in statistics?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">No, not only he is not an expert is also not from my field, which turned out to be an important aspect of going through a process like this one.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay. But you at this stage again, in your own mind, I\u2019m not guilty of anything. I need people to be in the room with me as I\u2019m going through this process. You\u2019re not really gaming out exactly what the elements of a defense would have to be. You don\u2019t even know what the nature of the charge is fully, so it\u2019s understandable you would pick those two.<\/p>\n<p class=\"para\">But it does raise the concern that if they don\u2019t turn out to be the kind of people necessary to help you defend yourself, you\u2019re stuck. You\u2019ve picked your two, you\u2019ve played your cards. Those are the rules, of course, not the rules approved by the faculty, but those are the rules that you\u2019re being forced to live under.<\/p>\n<p class=\"para\">I imagine you must have had conversations with the research integrity officer clarifying the fact that, for example, you can\u2019t go talk to the research assistants, you can\u2019t talk to other members of the faculty, you can\u2019t get help from other people that you really were stuck with&#8230; or stuck is a bad word, because there\u2019s both of them are perfectly decent people to do the job. They just weren\u2019t the only people, and they weren\u2019t all the people necessary to do the job. So you had them, and those were the people you could work with. You must have confirmed that understanding repeatedly.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">I did. And as you said, it became a real limit, because due to the way these studies are put together and the papers are written, you truly need other people to be involved, to try to understand or even reconstruct who did what and when for each of the studies in question.<\/p>\n<p class=\"para\">Yeah, yeah, I can imagine. Okay, so that\u2019s October 27. November 5, the process formally begins. The Business School appoints two faculty members, and their job is to investigate enough to decide whether a formal process should proceed. Where is Data Colada in all of this?<\/p>\n<p class=\"para\">It\u2019s nowhere since I only knew that the accusations against these four studies were made by an anonymous complainant. And so I didn\u2019t know that it was Data Colada.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And but now, do you know whether Data Colada had any role in these early iterations at the business school to determine whether, in fact, there was misconduct in this case?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">Now I do, and I know that they communicated with the research integrity officer multiple times.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And what was the nature of the communication to provide information or to get information?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">It was more to get information and almost dictate some of what the process should be doing, or what the committee should be doing in the way they looked at the data.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Yeah. And again, I can understand from Data Colada perspective, Data Colada thinks, look, we have these great blog entries, you know. And of course, a blog is the most important thing in the world, but we have these great blog entries, and we have agreed not to publish in exchange for you doing an investigation. So we have some stake in this investigation, and so we\u2019re going to be right there with you, and we\u2019re going to be describing what you should be demanding. To get the evidence to prove guilt or innocence. I\u2019m not going to assume they intended to prove guilt.<\/p>\n<p class=\"para\">But from a process perspective, that\u2019s pretty outrageous, because this should be the business school making its judgment, not the business school as the handmaiden of like these data vigilantes. You know, God bless them for their work. But that\u2019s not the appropriate role in this stage. Yet, that seems to be what\u2019s going on.<\/p>\n<p class=\"para\">So you\u2019ve got November, December and January. What do you know? What do you think is happening? Because nothing\u2019s happening directly with you. My understanding is you\u2019re just kind of locked in a room, not allowed to talk to anybody while this is all going on. Is that right?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">That is correct, and more than locked in a room, I was doing my job of teaching and continuing the research, which I think is an important aspect of all of it, since after the fact, some people looking back said, &#8220;Why did you keep working on your research?&#8221; And almost being angry that I did. But if you take my perspective of &#8220;I\u2019m part of a process, I know I\u2019ve done nothing wrong. I\u2019m sure that the process will prove that,&#8221; then what I ought to be doing is continue helping my students and continue pushing on the research.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Yeah, and you had a bizarre, I mean, from you know, a law professor\u2019s perspective, you had a bizarre amount of teaching obligations at this time, right? So describe like what you\u2019re teaching.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">I was teaching a new course that became a first-year course for the MBA students. So it\u2019s over a 1000s of them on inclusive leadership, and so it was a lot of work to create the materials. I was also chosen as the course head. That means that you\u2019re managing eight professors. Eight or more professors were teaching different sections of the course. So it was a very intensive period, from a teaching perspective.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay. And then you\u2019re also doing academic research.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">I\u2019m doing academic research. And then as a good HBS professor, you\u2019re also participating in all other activities, like promotion processes.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so you\u2019re a regular professor. You\u2019re teaching what is, from my perspective, an ungodly amount of number of students at the same time. But in the back of your mind, you know, there\u2019s this ticking time bomb of a process which, because you believe, because you know you\u2019re innocent. You don\u2019t believe it\u2019s going to blow up. You have faith in the system. And so that might not be debilitating to know that that\u2019s going on, but it can\u2019t have been pleasant.<\/p>\n<p class=\"para\">But so in February, seven months after these charges were brought to the Harvard Business School by Data Colada, in February, they finally talked to you about it, right? So describe what happens in February.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">So in February, we had a meeting. It was me, the two committee members of the inquiry committee. So these two Harvard Business School professors&#8230;<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Yeah, and I just want to be clear to the audience, I\u2019m not going to\u2026 I\u2019m asking Francesca, and I\u2019m not going to mention who these faculty members are. It\u2019s a hard and thankless task, the work that they\u2019re called to do, and I don\u2019t intend our conversation to burden them personally at all, so all we\u2019re going to do is describe what happens, and these anonymous people participating in the process will remain anonymous.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">I appreciate that, and, like you, I went back to the interview notes, to the transcripts, and I actually started by thanking them for doing what they were doing. Since I know the life of an HBS professor: it is very busy, and so doing this on top of that was something that definitely was calling for being grateful. But I was there. We were on Zoom, all the meetings with them.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Right, this is COVID.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">Yeah, yeah, and it\u2019s the two of them. Is me. I had my lawyer, and so in order for my lawyer to be there, there was the General Counsel of Harvard on the call as well, as well as the research integrity officer. I think that&#8230; that\u2019s it in terms of who was present. But, at that point, again in going back to the transcript, they asked some questions about my general practices. Again, in thinking about who the people there were, and maybe at the time I didn\u2019t quite understand it, but they didn\u2019t know my ways of working or my practices.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">They don\u2019t know the practices of behavioral science.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">Not to the level I think that we\u2019re talking about now.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Because obviously not every business professor, business school professor, works with data the way you work with data. Not all of them would have an intuitive understanding of the complexity of managing 30 research assistants to pull together the data for a project, right? So they could view it the way I would view it if it were a faculty member from the law school, like, here\u2019s some problems. You\u2019re the one who benefits. So I\u2019m presuming you\u2019re guilty here.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">Absolutely. I could imagine you looking at the data and truly not understanding why is it that you as a faculty don\u2019t own every step of the process, but when you put it in the context of multiple studies per paper, multiple papers a year, and the field is one of joint effort, and in fact, the joint effort is praised because it allows us for better research, then you\u2019re in a very different world.<\/p>\n<p class=\"para\">And so some of the questions were about understanding my practices, as well as going back to what the anonymous complainant brought to them, since they submitted a detailed letter, report, if you will, on December 3, and so they had the chance to see it, but I didn\u2019t. And so&#8230;<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">This is an important point. So, so what Data Colada submitted, and you understood it to be an anonymous complainant, was a detailed description of why they thought there was a problem with the with four papers.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">Yeah, I\u2019m going back with my memory, since this is an important detail, and I want to be accurate, I know that it exists since I saw it later, but I don\u2019t know if I had it at the time. But what I did have was from the committee, they would share their screen on Zoom and show me what they believed to be the anomalies that this anonymous complainant pointed out to them. And so, for example, for one of the allegations, they pointed to the fact that there were 20 rows (turned out to be 24) with Harvard written as the answer to one of the questions of the survey. And it seems strange, and so they were asking me to make sense of it. And if I go back to my interview, I didn\u2019t have good answers because I didn\u2019t really had the chance to try to make sense of the data.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Yeah, if you had manufactured the data, you would have had a pretty good answer, because you would have come in there thinking, &#8220;Oh, those are the 24 rows there must be they must have discovered the 24 rows I added. So what\u2019s my answer? Oh, here\u2019s the answer.&#8221; But you\u2019re kind of cold called, here are 24 rows that seem to have these weird characteristics to them. Like, what is this? And you don\u2019t have a good answer.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">That\u2019s right.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And then the question is, how do you, as investigators, read the fact that you don\u2019t have a good answer. And there\u2019s a lot of great work to demonstrate that we\u2019re not actually very good, necessarily, in reading the integrity of another person, depending on the character, depending on the manner of the person. But they\u2019re obviously listening to you, describe, give answers.<\/p>\n<p class=\"para\">As you left that meeting, it\u2019s the only meeting, I mean, leave at zoom you logged off of the zoom call, which is the only time you met with the two committee members during this investigation, the investigatory stage of this, did you feel anxious, more anxious or less anxious? Did you feel like this was a problem that was going to go away? Or did you feel like, wow, we don\u2019t really have a clear sense of what makes this go away.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">I think the emotion I remember having was one of sadness. It was the first time, and in fact, I commented on this in the interview itself&#8230; It was the first time I was meeting one of my colleagues, since one of the faculty members who was part of the Inquiry Committee, was a person I never had a conversation with up to that point. And so it just felt so sad and disappointing that that was the entry point to one of my own colleagues.<\/p>\n<p class=\"para\">Yeah. Okay, so February, there\u2019s this meeting. March happens, but you don\u2019t hear anything? What are you thinking?<\/p>\n<p class=\"para\">At that point, we were in the middle of finishing teaching this course, and so my focus really was on what was happening. And the course was&#8230; we struggled in getting it right, and I\u2019ve always been a really successful and effective teacher. And so it was the first time where the materials weren\u2019t as well received as we hoped. And so there was work to do, and so my attention was there.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">You were distracted, formally, fortunately. Okay, in April, the committee concludes that a formal investigation needs to happen. And they appoint a third committee member, and they begin their work. So they notify you of that through the through the research integrity officer?<\/p>\n<p class=\"para\">Francesca Gino They did.<\/p>\n<p class=\"para\">So the research integrity officer just gives you this conclusion or gives you a letter that says something?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">He always sends you a note that says, &#8220;I\u2019m about to share a confidential note about the process,&#8221; and then the note would follow.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">I see. Okay. So are you at home? Are you in your office when you read this?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">I believe I was in my office. And you might ask, how did that feel?<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Yes, I\u2019m going to ask how it did feel.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">It felt surreal. At the same time, I remember when back in late October of 2021, when I was interviewing the lawyers that the research integrity officer suggested, one of the consistent things that they said was, &#8220;look, the process entails an inquiry stage and then an investigation stage. Most, if not all of these cases end up in an investigation, and so the inquiry is almost something that we do to then lead into an investigation.&#8221; So my expectations were, an investigation is going to happen.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so this wasn\u2019t a surprise.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">That\u2019s exactly right.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Disappointment, maybe, but not a surprise.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">Mmm mmm [in agreement].<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay. April. May: the research integrity officer informs you that Harvard, the Harvard Business School, has hired a forensics firm, a firm called Maidstone. What\u2019s a forensics firm?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">So forensic firms are groups of experts who know a lot about data and they know a lot about digital evidence, and so they\u2019re people who can look at files and make sense of whatever evidence could be helpful. And in this case, in trying to understand what happened.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Are they cheap?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">I\u2019m told they\u2019re quite expensive. And now I know for a fact that they\u2019re quite expensive.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Because they\u2019re highly talented, like typically PhD type people who didn\u2019t want to be academics, but they wanted to do really consequential work. So one would think that learning that Harvard had hired a forensics firm for them, not for you, you might want to hire a forensics firm. Did you hire a forensics firm?<\/p>\n<p class=\"para\">I wanted to hire a forensic firm. It\u2019s one of the things that Gary suggested doing, and when I asked the research integrity officer, I was told I couldn\u2019t because I used up my two slots.<\/p>\n<p class=\"para\">You played your two cards.<\/p>\n<p class=\"para\">Okay, so this is a really critical point. And I say that as a member of the Harvard faculty, because it just strikes me as outrageous.<\/p>\n<p class=\"para\">You know, you\u2019ve effectively been gagged already. You don\u2019t have an opportunity to talk to the people who could help you put together the evidence you would need to establish what I believe is true. And you\u2019ve asserted convincingly to me that there is no fraud here. There\u2019s no misconduct. You\u2019ve already been gagged and made that much more difficult. But they think it\u2019s necessary to have an expert to understand the data. The idea that they don\u2019t let you have an expert to understand the data is extraordinary.<\/p>\n<p class=\"para\">You know, it would be a violation of due process in a court, the idea that one side gets special access to a technology to understand the evidence in the case, but the other side is blocked from it and blocked from it based on the fact that you played your cards already. I mean, it\u2019s not like when you decided those two, the research integrity officer said, &#8220;Oh, look, you know might be you want to reserve one of these because you\u2019re going to need to have really high powered experts to evaluate data. So make a good choice here.&#8221; You had no clue about what was going to be necessary here.<\/p>\n<p class=\"para\">And yet, here we are. You are blocked from being able to have an expert who could help you resist or rebut or interrogate the evidence that comes out from the expert on the other side. And of course, that becomes as we, you and I are not going to talk through this, but in talking through of the four charges, that becomes quite significant, because eventually it\u2019s shown this expert\u2019s work is problematic, and they withdraw the expert as the expert they rely on in the tenure revocation proceeding.<\/p>\n<p class=\"para\">And the errors are errors that if you had had an expert before they concluded you were guilty, you could have pointed out to the committee, that they were expert and there are errors in the analysis. So it\u2019s not just that it was wrong. It was actually harmful to the process of coming to a conclusion of whether you had committed academic misconduct. And so, you know, as I look at this, this feels to me like the most significant point at this stage of the process.<\/p>\n<p class=\"para\">Now, to be perfectly transparent, the research integrity officer says that he didn\u2019t forbid you from hiring an expert. I mean, Gary, your advisor told you should hire an expert. I would imagine that if he told you, you would agree you needed to hire an expert. The fact that you didn\u2019t hire an expert was not that you were, you know, lazy or not interested or not concerned. There must have been some reason why you didn\u2019t. So that seems circumstantially to suggest that, in fact, you were told.<\/p>\n<p class=\"para\">But the research integrity officer is quite clear that he never told you you couldn\u2019t hire an expert, that you were free to hire an expert, that there was this hidden exception to the gag order. The gag order says you couldn\u2019t talk to anybody except the expert, which you were allowed to hire but did not hire. When you heard the research integrity officer say that, what did you feel?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">It\u2019s hard to hear\u2026 looking back at the history and changing the facts in the moment where the research integrity officer told me that I couldn\u2019t hire a forensic expert. What I told myself, well, it\u2019s going to be a fair process, and so I\u2019m sure that their expert is just going to figure out the truth and explain what they see in the data. And so I was starting from the standpoint of thinking that the committee and the research integrity officer would do their work with this expert to simply prove that I didn\u2019t do anything wrong.<\/p>\n<p class=\"para\">At the same time, in May, the research integrity officer told me that the committee intended to interview the various RAs involved in the research. And so again, I\u2019m there thinking they\u2019re gonna have a lot of information and evidence to simply come to the conclusion that there is no evidence against me.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so you\u2019re still operating with the belief that everybody\u2019s operating in good faith. And I\u2019m not saying they weren\u2019t operating in good faith, but you\u2019re not anxious about it, because, again, you think that the process is going to prevail the truth. And the truth is you haven\u2019t engaged in academic misconduct.<\/p>\n<p class=\"para\">But I\u2019m asking a different question. I\u2019m saying at the moment you heard the research integrity officer say that he didn\u2019t tell you you couldn\u2019t hire another expert. What did that feel like?<\/p>\n<p class=\"para\">It felt awful. Here you have the research integrity officer of a really important institution where Veritas means something, and he lied under oath. That was really hard.<\/p>\n<p class=\"para\">Yeah, it\u2019s hard for me even just to hear it. And this is at a stage where you can understand, maybe you know, this turns out to have been an important decision. And clear to everybody, it was the wrong decision to forbid you to hire a forensics expert. So maybe there\u2019s a motivated reason why his memory evolved to believe this, you know. So he maybe didn\u2019t actually lie, in the sense that he actually did believe he said this to you. But at the moment when this was operative, your advisor is telling you you need a forensics expert, and you believe you want to hire a forensics expert, you don\u2019t hire a forensics expert. It\u2019s pretty hard to believe you did that just because you believed it was not necessary, you\u2019re not listening to Gary.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">And for anything important, really critical that the research integrity officer suggested I should do, there is an email when he suggested lawyers. There is an email with names of lawyers to try and speak to. And when it came to a forensic firm, there is no email whatsoever.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, that\u2019s May. In June, coming up on a year since this whole thing began, Data Colada begins to get impatient. In the end of June, which is just at the year point, they send the research integrity officer a letter demanding to know when the process is going to complete, and quote &#8220;whether and when they should take matters into their own hands.&#8221; So again, they\u2019re kind of holding the gun to the head of Harvard saying you got to resolve this because we need to publish our blog posts. So the committee is obviously feeling some pressure. From their perspective, nothing\u2019s really happened. From Data Colada\u2019s perspective, nothing\u2019s really happened, and they\u2019re acting to make their threat to HBS play.<\/p>\n<p class=\"para\">Okay, so Francesca, it\u2019s the summer of 2022, one year after the charges have been raised, about four of your papers, HBS is investigating these charges. You\u2019ve learned HBS gives its forensics firm data from your computer for them to evaluate.<\/p>\n<p class=\"para\">Okay, let\u2019s pause on this point, another really critical point for just a second. They have your computer, right? You told us that you gave them your computer when they told you that you were being investigated and when they took it from you. Do you know what they did to it?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">When I showed up on that October day, I showed up with all my HBS machines, and so there was an IT person who was supposed to take forensic images of my machines so that they would have all the evidence needed to make sense of these allegations.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Yeah. So to be clear, this is not something that\u2019s obvious. You can take what\u2019s called an image of the machine, which is more than an image of the hard disk,1 it\u2019s an image of the state of the machine at a particular time. And that image of the state of the machine kind of draws the baseline, and it\u2019s from that image that you make a determination of what happened on that machine. Because obviously essential to the case is them establishing that you manipulated the data in some way. So if you did, there might be evidence on the machine that you manipulated data in some way. There might be files in the system that would help reveal that. It reveals what web pages you\u2019ve opened, what thumb drives you\u2019ve inserted, all that sort of data is there.<\/p>\n<p class=\"para\">Indeed, you could say this is kind of data forensics 101. That if you\u2019re going to know how a machine was used, the investigator must take this image at the moment the investigation is launched, because that image permits the analyst to determine every fact about the use. It records the websites. It includes system logs. This is the critical part, right? That could reveal when and how files were created, maybe even include remnants of deleted files, because when we delete a file, it doesn\u2019t actually delete it, it just deletes the entry in the file list. So the problem is that that information is not stored forever, because these system logs have routines to erase them after a period of time. Number of days after they\u2019re created, they get erased depending on which log it is, so it doesn\u2019t take over the whole of the computer\u2019s hard disk. But you were told specifically that the research integrity officer had made forensic copies of the devices, but he had not. He had not taken forensic images. All he did was have a 1 Instead of saying &#8220;more than an image of the hard disk,&#8221; it would have been better to say &#8220;more than copying some files from the hard disk.&#8221;<\/p>\n<p class=\"para\">computer technician turn on the computer and copy certain files from your computer onto another computer, doing nothing to preserve the evidence necessary to know who did what when. Is that right?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">That\u2019s exactly right.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And you didn\u2019t know this at the time obviously, right?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">I did not.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay. So when HBS gives Maidstone, not a forensic image of your computer, but instead, copies of files from your computer, Maidstone is working with an imperfect set of evidence, right?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">Yeah. So it\u2019s, in a sense, error upon error. Instead of taking a forensic image, they copied files, losing some important evidence&#8230;<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">the metadata evidence&#8230;<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">That\u2019s exactly right. And then they go to their expert, and rather than giving them all the evidence so that they could work from a complete set, they gave them, I believe, 343 files out of one terabyte of data. And so what Maidstone was working with was very incomplete.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So Maidstone not only had imperfect data, because the metadata wouldn\u2019t have been preserved in the way it would have been if it had been an image of the computer, they didn\u2019t even have all the files. But with that subset, they begin to do the work.<\/p>\n<p class=\"para\">So over the summer, the committee, now it has three members, also begins to do its work. That work was supposed to involve interviewing the people involved in the process of you producing your work like that\u2019s what we\u2019ve been talking about during this conversation, because obviously there are a lot of people touching data and to figure out how certain things might have gone wrong, would be good to talk to those people.<\/p>\n<p class=\"para\">So you had identified 12 RAs that had been touching the work related to these four papers. How many of those 12 did the committee decide to interview?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">The committee interviewed only two of them, and one was an RA who worked on two of the papers, but the committee only discussed on one of the papers.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">&#8230;with that RA.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">Mmm mmm [in agreement].<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so two out of 12. And did you know that they were interviewing just two out of 12?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">No, the research integrity officer told me that they would be going out to the RAs to truly understand who did what.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And so you believing in the system, believe you talk to all the RAs. The RAs are not going to say that Francesca is a fraudster who\u2019s been trying to manipulate data. They\u2019re going to describe the process. They might help the committee understand just how messy and complicated the process is.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">And Larry, I think it\u2019s important to note that it\u2019s not that I was in constant communication with the research integrity officer. Just the opposite. It\u2019s not that when the committee went and interviewed an RA, they would come to me and say, &#8220;one of the interviews has been completed.&#8221; I would only hear from him under very specific milestones: a report is available or a meeting is coming up. And so you\u2019re almost feeling like, you\u2019re sitting waiting for something to happen, and when that thing happens, you try to prepare to respond to whatever question is coming your way.<\/p>\n<p class=\"para\">Yeah. And just to be clear, you didn\u2019t talk to these 12 RAs either.<\/p>\n<p class=\"para\">I did not.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Because you were not allowed to.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">That\u2019s exactly right.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And the research integrity officer doesn\u2019t say that there was some hidden exception to the gag rule that said you were allowed to talk to the 12 during this process. Everybody agrees we\u2019re not allowed to talk to them.<\/p>\n<p class=\"para\">Okay. So now we\u2019re into the fall of 2022. At some point, Maidstone completes its first report. HBS gives the report to you. Can you describe it? Do you remember that first report?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">These are dense reports. They come in Word format, with printed tables and different colors that try to explain the data. And having now had the opportunity to go through each of them very carefully, also with the help of forensic experts&#8230;<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Not at the time.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">Not at the time. This is after the investigation, after I was put on leave&#8230; it takes hundreds of hours to just try to understand what the forensic firm actually did, to figure out whether they made mistakes, and then for you yourself, doing the work, trying to understand the data.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Yeah. And so you had a limited amount of time to review this report before you were to meet with the committee to discuss the nature of the report, right? So you had, was it two weeks?<\/p>\n<p class=\"para\">Yeah. So they the research integrity officer sent the various reports separately. So I believe that the forensic firm sent them to the committee, the committee might have had extra questions, send them back. And when they believe they were final, they discuss them as a committee, then send them to me. And they were doing them in sequence, so I didn\u2019t receive them all at the same time. And it\u2019s now September and October, and again, we\u2019re back into teaching. And for one of the reports, I only had two weeks to look through it. And so this is hundreds of pages with very detailed information.<\/p>\n<p class=\"para\">With very detailed information and no expert help, because you\u2019ve got to understand it and be able to evaluate whether what these data experts are describing is plausible or true, or have some sense of what you would do to resist it. So you meet with the committee. All three of them. Is this again a Zoom meeting?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">It is again a Zoom meeting.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay. And this is the only meeting you had with this committee, right?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">Yep.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And so how long did this meeting go on?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">I believe it was three hours. Well, the afternoon of Monday. Initially, the committee had decided to have two separate meetings in light of how dense the reports were, I believe. And the meetings were scheduled before I received the reports. And so when the time came very close and I still had no reports, I asked for the meeting to be moved so that I would have time to have a look at it, and then the committee didn\u2019t want to push the meetings till after Thanksgiving, and so we ended up with only one meeting, making it a little longer.<\/p>\n<p class=\"para\">Okay. And what kind of questions did the committee ask you in that meeting?<\/p>\n<p class=\"para\">It is hard, because what a lot of the meeting was about is them screen share part of the analysis that Maidstone had done asking questions about, how would I explain some of what they were seeing in the data. And so when I look back and I look at the transcripts, I almost feel ashamed of how I answered. But it came from a person who&#8230; you can\u2019t, on the spot, make sense of data that another person collected and clean, and you didn\u2019t have the chance to fully understand prior to having the question being asked.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Right. And this is important, because ordinarily you would think you know if you\u2019re being asked about your behavior &#8211; like you drank a bottle of vodka and you got down, you went down and got in your car and drove&#8230; Like, you know, everything about what happened in that story. But they weren\u2019t asking you about, you know, did you drink a bottle of vodka? Did you get in the car and drive?<\/p>\n<p class=\"para\">They were asking you about an evaluation that\u2019s made by some expert, who\u2019s displaying an expertise you don\u2019t have of data that was on your computer to reach a conclusion that it demonstrated you must have engaged in fraud or not. You are not in a position to make that evaluation right, because that\u2019s not your expertise, right? But that\u2019s what they\u2019re essentially asking you to do.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">Yeah, looking back, I think the entire structure of this interaction was just wrong for the purpose of trying to understand what happened in the data. I think the right structure of this interaction is to truly sit down with the experts and actually going through Excel, going through the files rather than responding to something that you see on a screen.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">yeah, yeah, I can\u2019t imagine how you would begin to respond to that. And is the lawyer there with you?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">Yes, so the lawyer was there as well as Harvard General Counsel.<\/p>\n<p class=\"para\">All right, so you left that meeting, or the meeting logged off.<\/p>\n<p class=\"para\">Were you more concerned than you had been before?<\/p>\n<p class=\"para\">I remember crying pretty hard&#8230;<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Is this the first time you were crying?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">No. But I cried because, again, these are your colleagues, and I think as scholars, one of the things that we might not do exceptionally well is to say, &#8220;I don\u2019t know.&#8221; And I felt bad that I couldn\u2019t give them the explanations they were looking for. And so I cried.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Yeah. Okay. So a number of weeks later, how much longer is it until you get a draft report?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">It\u2019s mid December.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay. So three weeks later, four weeks later, you get a draft report. The research integrity officer gives it to you, what does he say is going to happen now, when you have the draft report?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">Receiving the draft report was hard, because in the draft report, the committee was taking a stance in terms of their belief about what the anomalies were, data falsification. And in addition to that, they were taking a stance in terms of who was responsible for data falsification.<\/p>\n<p class=\"para\">And it was hard to take, because in reading the Maidstone reports, one of the things that Maidstone did was just trying to understand, when you compare different files, an earlier version and a later version, if there were differences, and found that there were, in fact, differences. And they didn\u2019t ask themselves the question of, how do we explain them? How is it that they could have come about? And they certainly didn\u2019t make a determination on who was responsible. And yet the committee had reached that conclusion.<\/p>\n<p class=\"para\">And so I remember, again, it\u2019s the end of the school semester for my kids, I remember participating in a play, and before the play started, I called Alain. And I was in tears because I couldn\u2019t believe what I read in the report, and I started asking for time so that I could respond to the draft report.<\/p>\n<p class=\"para\">And I remember Alain saying that he didn\u2019t believe that there would be changes to what the committee had concluded, and in fact, he recommended that I seek mental health support because it was going to be hard.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Wow. So you\u2019re at your kids\u2019 school. We didn\u2019t introduce your kids. Your kids are beautiful and young. So how old are they at this point?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">At the time, I had a four-year old, a six-year old, a seven-year old and a 10-year old.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Wow. But you throw yourself into writing a response. For six weeks you write a response to this report.<\/p>\n<p class=\"para\">I\u2019m not sure I would have gotten over the first point, which is the evidence you have is of a difference in the files. You don\u2019t have evidence that the computer was used to make the difference, you don\u2019t have the system files necessary to demonstrate that so and so at this point, tapped this and therefore got that. You don\u2019t have any of that evidence. You\u2019ve instead, just inferred from the fact that there\u2019s a difference, that you, one of 10 people writing this essay are responsible for the difference.<\/p>\n<p class=\"para\">And I\u2019m not saying, as some people have suggested, that no, no, there\u2019s some evil research assistant who\u2019s trying to screw you. No, it\u2019s just that there\u2019s a difference. And there\u2019s any number of reasons why there could be a difference. And the job of the committee must have been to decide that they had enough evidence to say you made the difference, not that they could infer it, but that they had evidence of it. And so for six weeks you spend trying to respond. Who was helping you in drafting that response?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">It was Gary&#8230;<\/p>\n<p class=\"para\">Again, a great professor, but not an expert, to respond to Maidstone.<\/p>\n<p class=\"para\">And in fact, at that point, his role, I think, became one of recognizing that it was hard to respond to what the committee had produced. And so it would be me often going to Concord and sit down in his office drafting while he was working on something else, and when I had questions, I would ask him for his counsel.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So how long is your response?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">My response was, I believe 35-36 pages, but then I had a lot of exhibits, and so the document became significantly longer, so we\u2019re up to 250.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, and when you\u2019re finished with that work that you\u2019ve done for six weeks, do you feel good about it?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">I remember almost having a little celebration in terms of saying, &#8220;Okay, this is going to be fine,&#8221; because what I had done is looking back to the Maidstone reports, at least call out what Maidstone had concluded, and referring to the overall evidence, and sort of suggesting that the committee didn\u2019t have the evidence to conclude that there was data falsification and that I was responsible. And so I actually felt good.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Yeah, you felt you\u2019d achieved what you needed to achieve. So you turn your report in, and how many weeks later does the committee finish its final report?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">So it\u2019s February 17, 2023 when I submitted my response, and it\u2019s early March when I hear that the report is final.<\/p>\n<p class=\"para\">So they take less time than you took, and they conclude that their report is final. Their final report is, as the research integrity officer said, basically, their draft report. Did the committee tell you this? Or how did you understand this? They decided not to incorporate responses saying that they were not germane. Somebody says that in this context, I think they just don\u2019t know what that word means like, in what sense could they not have been germane?<\/p>\n<p class=\"para\">I am not sure. I thought that they were relevant, but they weren\u2019t\u2026 well, according to them.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">They didn\u2019t see they were. Yeah, Maidstone had found discrepancies, differences in the files. They hadn\u2019t said who was responsible. The committee concluded they knew who was responsible. You tried to respond by saying, actually, you don\u2019t have evidence, either they\u2019re intentional or that I did them. They didn\u2019t think that was relevant.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">I think what the committee was really trying to get at, or that they wanted, at that point, was clear explanations of how the anomalies came about, and at that point, not having had the forensic help that I needed to answer that question, I didn\u2019t give them that answer.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And that\u2019s a great point, because later, we\u2019ll see in the next episode, that when you did have an expert, you actually were able to explain how the anomalies came about, at least give an account that competes with and in some cases, I think, completely negates the suggestion that it was done intentionally. But you didn\u2019t have the expertise at this stage to be able to craft that kind of response, and so you couldn\u2019t give them what they were eager in getting.<\/p>\n<p class=\"para\">Okay, so let\u2019s just dig down and unpack this a bit, because I think it\u2019s important to hold it all on one page. Everybody understands there were differences. Everybody acknowledges that there are many people touching these data, and many people who could have created these differences, but they concluded it must have been you, because only you benefited, they thought, from the differences.<\/p>\n<p class=\"para\">Now we\u2019re going to see later that in each case, there were differences that both strengthen the conclusions of your paper and weaken the conclusions of your paper. So even here, the claim of motive is weird, but we\u2019re going to focus on that later. There were differences. They concluded it must have been you.<\/p>\n<p class=\"para\">Now I, you know, have said, I think in some contexts, that\u2019s a fair conclusion. Just about the same time, there was a right-wing blogger who was attacking Harvard\u2019s president, Claudine Gay for plagiarism. The charge was that in 8 of the 17 works reviewed, there were 50 incidents where the instances where there are passages copied from other people\u2019s work without attribution or quotation marks, Harvard eventually concluded they were not intentional. But no one would doubt that she was responsible for the errors in the sense that she\u2019s the one who typed them, like it\u2019s not like a research assistant typed them for her.<\/p>\n<p class=\"para\">But here they both had to conclude that you were responsible for the errors, and that the problems that they had identified were intentionally created by you. And that\u2019s what they asserted in concluding you were guilty. Now, when you read this, you must have been devastated.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">I think that that\u2019s a good word for it, since the process had been hard and I just couldn\u2019t understand why the conclusion that they reached was the conclusion that they reached. And I couldn\u2019t make sense of it.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">But you were finally understanding the consequence here.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">Yes.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Like this was not a process that was reaching a happy ending by getting to the truth, from your view of the truth, and you know more than anybody knows what the truth is. So you finally now are confronting the fact that this has exploded in a way that\u2019s going to be catastrophic for you.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">Yes, I&#8230; At the time in March, I felt devastated, but I was still hopeful. I thought that, in the end, what the committee was doing with their work is making a recommendation to the Dean, and the Dean would look at the report and see all the evidence from the interviews with my co-authors who spoke about my integrity or the way I worked. And that he would understand that maybe more work needed to be done, or maybe there would be consequences or extra training in terms of how I need to manage my lab, handle data&#8230; but I don\u2019t think I thought at that point that I would be put on leave.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Did others suggest you would be? I mean, you only had two people in your orbit, your lawyer and Gary.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">My lawyers were surprised, and they kept saying that they had not seen anything like that in other parts of Harvard. And they were so surprised that they decided to write a letter that they hoped would go directly to the Dean of the Business School. But being lawyers, they sent the letter to the General Counsel, making some important points about the fact that they didn\u2019t believe that the committee had the evidence to reach the conclusion that they did, and making the case for the conclusions to be changed and for the sanctions to be different.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">But we don\u2019t think the Dean got that.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">Mmm mmm [indicating no, he did not.]<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so the Dean didn\u2019t have that. The Dean on the basis of the charges made by Data Colada and the investigation committee, but a report that didn\u2019t include your responses but reached the conclusion that you had intentionally committed academic misconduct, concluded that you needed to be removed.<\/p>\n<p class=\"para\">And removal is quite significant because it also means you don\u2019t have pay, you don\u2019t have health care, you don\u2019t have any of the ordinary support that in American society we depend on. On June 13, three months after the committee had made its conclusion, the Dean calls you into the office. Describe what happened.<\/p>\n<p class=\"para\">I want to describe what led to that meeting. Since, again, after March, there was silence. So I\u2019m there waiting, not knowing what is happening. And as we got into early June, I sent a note to the Dean\u2019s office, sort of asking whether I could go on vacation as planned, knowing that this was hanging. It\u2019s this strange reality where you have a sense that things are happening behind the scenes, but nobody is telling you what exactly is happening. And so it had been a hard few years. And so, for the first time, we had a vacation planned as a family on June 14.<\/p>\n<p class=\"para\">And so you\u2019re going to Italy, I believe.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">I was going actually to the islands. So we were going to the beach together in June. And I reached out. I was hoping to go and be offline, and so I shared my schedule with the Dean\u2019s office. And so they said, in light of your schedule, let\u2019s meet on June 13.<\/p>\n<p class=\"para\">And it was a day where I was teaching the entire day in executive education, and I remember actually sitting at 5pm in the cafeteria, waiting for the 5:30 meeting with the Dean, and&#8230; I was just planning scenarios. And I thought that maybe there would be conversations about, &#8220;what do we learn from this experience and what needs to change?&#8221; To &#8220;sorry you had to go through this clearly the wrong conclusion.&#8221; And I\u2019m not even sure if I truly thought through the possibility that I would be asked to leave.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">I don\u2019t think you were asked. So he brings you into the office. It\u2019s 5:30 June, 13. He tells you not to speak, and he has a letter that he reads to you, right?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">Yep.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Is he emotional? Is he uncomfortable?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">He doesn\u2019t look uncomfortable. He doesn\u2019t look emotional.<\/p>\n<p class=\"para\">And this letter is telling you you have been put on leave, unpaid leave. And the research integrity officer then contacts the journals that have published these pieces and shares the Data Colada data and the Maidstone reports and the conclusions. I take it he didn\u2019t share your responses.<\/p>\n<p class=\"para\">He did not.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Yeah. And then the Dean reaches out to one of your colleagues, a professor, and asks that colleague to counsel you out. What does that mean?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">That means that the colleague came to me with a recommendation that I resign. And so the deal, in a sense, as it was posed to me, is that if I resigned, this would go quiet.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">You didn\u2019t take that deal.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">I didn\u2019t take that deal. As I told my colleague, I didn\u2019t commit what I\u2019m accused of.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So it didn\u2019t go silent. Which means that four days later, on June 17, the news goes to the public.<\/p>\n<p class=\"para\">I remember hearing it and I was astonished. You know, I didn\u2019t believe it. But it almost writes itself, the story, right? You know, an academic studying the psychology of fraud, convicted of engaging in fraud. I mean, the internet loved it. It was everywhere. Everyone was sure that, you know, this powerful Harvard professor, one of the young superstars of the Harvard Business School, turns out to be, as many think, not what anybody believes. And so that, like blows up. Data Colada publishes its report on its blog. You begin to feel the real consequences of this. So what are the consequences beyond losing your employment with Harvard?<\/p>\n<p class=\"para\">It was really hard. I think that summer was full of great low points, because first it was telling the kids. They show up at 6am in the morning on June 14, ready to go on vacation. And we obviously didn\u2019t go on vacation. And we had decided which, in retrospect, was possibly not a great idea to tell them that I lost my job. And so that was hard. And now I have four small children at home. And the press is going on fire, and reading the stories was really, really hard.<\/p>\n<p class=\"para\">I\u2019m an endless optimist. And so I remember at the time looking at my husband and saying, &#8220;It\u2019s okay, we are going to navigate through this, and I\u2019m going to be able to keep my consulting.&#8221; And I was wrong. And what he saw pretty much every day is a client or somebody I was working with calling up and saying that in light of the news, despite the fact that they didn\u2019t believe what they read, I was radioactive, and so they couldn\u2019t keep working with me.<\/p>\n<p class=\"para\">At that point, my husband had a job that he liked, but he was not a well-paid job since we had decided to focus on my career. And so it was really hard to realize that I had been the breadwinner for the family, and that it looked like I wouldn\u2019t be able to continue providing&#8230;<\/p>\n<p class=\"para\">And I was being attacked on one of the most important values that I hold.<\/p>\n<p class=\"para\">I reached out to you.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">You did.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">We went for a walk in a park. We\u2019re getting coffee at Darwin\u2019s.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">You had long white hair.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Yeah. And I was not surprised by what I felt after I spoke to you, because it\u2019s what I believed. So maybe it\u2019s confirmation bias, but I was convinced that there was an injustice that we had to figure out how to fix. There was a lot more to be done.<\/p>\n<p class=\"para\">And Larry, what was also difficult in this moment is that the timing of it was awful, because it all happened so fast. I arrived home, I think at 8 pm that June 13, and then again, I was worrying about, what do we tell the kids? How do we relate this to them?<\/p>\n<p class=\"para\">And I have 120 co-authors, I have students, I have RAs, and it just felt awful. It seemed like impossible that I would be able to reach them all. And so I called a few, but I couldn\u2019t reach everyone.<\/p>\n<p class=\"para\">Francesca, thank you for fighting. Thank you for this conversation. We\u2019ll continue the next conversation through the next stage that gets us to the end of the tenure revocation process. And then I will be talking to someone else to unpack the four charges and demonstrate why they don\u2019t support the claim that you committed academic fraud.<\/p>\n<p class=\"para\">There are many like me out there. We will always be here.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">I really appreciate you having the courage to tell the story. And this is a story of compounding error and feeling like the right things are happening too late. But I hope it\u2019s not too late, and that people are going to be open to hearing a side of the story that unfortunately I haven\u2019t been able to tell so far.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Thank you.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">Thank you.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">That was the first half of the story. From the beginning in July 2021 until the Harvard Business School determined in 2023 that Francesca had to be removed from the business school.<\/p>\n<p class=\"para\">The next episode will take the period from her removal until her removal from the university, having her tenure withdrawn by the President in 2025. Stay tuned for that episode.<\/p>\n<p class=\"para\">This podcast is produced by Josh Elstro of Elstro Productions. It is not affiliated with Equal Citizens. It is my own Larry Lessig podcast produced for him with his money, but not a lot. Josh is an efficient editor. You can subscribe to this podcast. Wherever you get podcasts, you can share it. I hope you do. There\u2019s a Substack you can find associated with me about the Gino case. That Substack will include not just these podcasts, but also materials that will help you understand the elements of the story.<\/p>\n<p class=\"para\">After this podcast gets posted. I will be posting the appeal, which I wrote to the President to get the President to reverse the decision or not to accept the decision of the committee that determined her tenure should be revoked. That will get ahead of the story just a little bit, but it might help in following the episodes that come after this one.<\/p>\n<p class=\"para\">Stay tuned. Thanks for listening.<\/p>\n<\/div><footer class=\"ep-footer\"><a class=\"ep-footer-link\" href=\"#gino-index\">\u2191 Back to index<\/a><a class=\"ep-footer-link next\" href=\"#gino-episode-3\">Next: The Hearing &amp; Tenure Revocation \u2192<\/a><\/footer><\/section>\n<section class=\"episode\" id=\"gino-episode-3\" data-screen-label=\"03 The Hearing &amp; Tenure Revocation\"><header class=\"ep-header\"><div class=\"ep-eyebrow\">Episode 03 &middot; The Law, Such As It Is &middot; Season 3<\/div><h2 class=\"ep-title\">The Hearing &amp; Tenure Revocation<\/h2><p class=\"ep-blurb\">The two-year process that ended in the first revocation of tenure in Harvard&#8217;s history.<\/p><\/header><div class=\"ep-body\"><div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para has-dropcap\">This is Larry Lessig. This is the third episode of the third season of the podcast &#8220;The Law, such as it is.&#8221; This season is considering the tenure revocation of Francesca Gino from the Harvard Business School.<\/p>\n<p class=\"para\">In the last episode, Francesca and I discussed the first part of the two-part process that eventually removed her as an HBS professor. That first part was the process run by HBS to determine her guilt. I said that first part was flawed. Francesca was gagged. She had no opportunity to build an effective defense, and while HBS relied heavily upon a forensic report produced by a company called Maidstone, Francesca was denied the opportunity to hire her own forensic analyst. HBS thus determined her guilt before she was given any meaningful opportunity to build a defense.<\/p>\n<p class=\"para\">We know this flaw in the process was hugely consequential, because once Francesca was free to hire a forensic firm, and once that firm had had an opportunity to study the Maidstone reports and demonstrate the errors in those reports, Harvard effectively withdrew those reports. HBS had thus determined her guilt based on a flawed report. Indeed, members of the committee expressly stated they had relied upon this flawed report in reaching their conclusions. Francesca had thus been denied the opportunity to show the report was flawed before it was used to condemn her to everything else that would follow. Everything terrible followed from this error.<\/p>\n<p class=\"para\">Let\u2019s call this fundamental error number one, because once this error was made and she was determined to be guilty, then all the resources of the Harvard Business School were deployed to defend this flawed determination of guilt. HBS hired the law firm of Ropes &amp; Gray, a powerful Boston firm that deployed at least five lawyers to defend the HBS determination. That defense is now the topic of this second part of the story, from the moment the Dean notified Francesca that she was to be removed until the university confirmed that determination and took away her tenure.<\/p>\n<p class=\"para\">Stay tuned.<\/p>\n<p class=\"para\">Francesca, thanks for continuing this conversation. So the last time we\u2019d stopped at the point in the story where you had learned from the business school that it had concluded that you\u2019d engaged in academic misconduct. The Dean called you into his office to tell you that, and he asked a colleague of yours to &#8220;counsel you out,&#8221; which means what?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">It means asking a colleague to come to me and say, leave, don\u2019t fight.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Don\u2019t fight, but you are innocent, so you are not going to leave without a fight. This is June 13, 2023. Later that month, the Dean asks the University to begin what\u2019s called the Third Statute proceedings, which sounds so archaic. Do you actually have any idea what &#8220;Third Statute&#8221; means here?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">I had to look it up. And I think it comes from very old universities that had set of rules for the way they think about their governance and discipline.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So this is the Third of the Statutes exactly in our rules. And of course, it\u2019s been around since the beginning. But has it ever been used to actually revoke somebody\u2019s tenure before?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">No, as far as I know, I\u2019m the first person who\u2019s ever gone through this process at Harvard.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And so I think you got a sense, a little bit of a clue of why that might be. Jumping ahead in the story a little bit. In January, so six months later, the lawyers representing you in this Third Statute procedure met with Harvard right and what happened in that meeting?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">We told Harvard that the committee got it wrong, that there were errors in the Maidstone analysis, and we wanted the chance to present our case, to discuss the merits of my case. And Harvard indicated an interest to just pay me to go away, and that involved me resigning. I decided again that I wouldn\u2019t resign.<\/p>\n<p class=\"para\">But this kind of gives you a clear sense of what actually happens whenever there\u2019s a fight like this. There\u2019s a meeting, they offer you a little bit of money, and they say, in exchange, you\u2019re just going to not fight any of the things we\u2019re saying.<\/p>\n<p class=\"para\">Yeah, I think there is the belief that Harvard has deep pockets and wide networks, and that if you decide to fight, it\u2019s going to be a hard fight.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Yeah, I guess you\u2019ve discovered that that\u2019s in fact, true.<\/p>\n<p class=\"para\">Okay, so you chose not to resign, going back now again. So, in June, HBS asked them to begin the proceeding to end your tenure. At the end of July, you received a one-page letter from the President, President Gay telling you that a complaint had been lodged based on the research misconduct proceedings that HBS had conducted, and thus they were initiating this tenure revocation process. But then you didn\u2019t hear again from them for a couple months, right?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">That\u2019s right, I only heard from Harvard again on October 25. I received an email from President Gay with the Third Statute complaint. And the email was very brief. It was, I think, six sentences long, and it indicated that a Hearing Committee would be convened.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">To review whether your tenure should be revoked?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">Exactly so.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">The 25th was a significant day in the life of Harvard, at least in the life of the President, because the 24th of October, Harvard had learned something about President Gay. Is that right?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">Yeah, they had learned that Gay had also been accused of academic misconduct.<\/p>\n<p class=\"para\">Yeah, so I imagine you\u2019ve gone through this process. You wouldn\u2019t wish it on anyone or anything like this process on anyone. But, of course, her process turned out not to be quite the process you\u2019ve gone through. Harvard learns of the charges that were made by the New York Post on the 24th of October. Within eight weeks, they had had two separate investigations to determine that while there were multiple instances of her failing to cite work that she had relied upon, for example, by paraphrasing the work or including it without any citation at all, in their view, the mis citations were neither &#8220;reckless&#8221; nor &#8220;intentional,&#8221; and therefore did not constitute academic misconduct.<\/p>\n<p class=\"para\">But the timing here was unfortunate because it raised an obvious question, right? You later discovered that at the time, President Gay determined to bring the Third Statute procedure against you, she declared, &#8220;We have to make an example of this woman.&#8221; Of this woman, meaning you, maybe also hoping it would be a kind of distraction from what was also going on at the time with her.<\/p>\n<p class=\"para\">Now, I think what\u2019s important about this is it just underscores how consequential the flawed HBS procedure was to you and to your case. Because obviously, based on that flawed HBS procedure, the President had decided that you were guilty. And you know, I have no problem with the idea that you\u2019re going to make an example of somebody who has committed academic fraud, but only after you really determine whether they\u2019ve committed academic fraud. But here that determination had been made on the basis of a report that Harvard would subsequently effectively withdraw. So President Gay, of course, couldn\u2019t know that, but her determination to make an example of you flowed from this flawed Investigative Committee determination.<\/p>\n<p class=\"para\">But shortly after that, in January, President Gay resigns from the presidency. Maybe it was a promise by her to resign that led the committee to limit its judgment against her. We don\u2019t know that. But despite the clear examples of her using the work of others without acknowledgement, she remains a member of the Harvard faculty today without any Third Statute proceedings against her.<\/p>\n<p class=\"para\">So this Third Statute proceeding was to have three stages. HBS would file a complaint. That complaint was to substantiate the charges made against you, the charges that would constitute academic misconduct. You then were to answer that complaint, and then there would be a hearing by a faculty committee, the Hearing Committee, about that complaint and that answer.<\/p>\n<p class=\"para\">Let\u2019s talk a little bit about this hearing committee. How big was the committee?<\/p>\n<p class=\"para\">There were seven members. I was told that it was supposed to be eight, but some HBS professors felt conflicted, and so it ended up being seven people.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">I don\u2019t think we should talk about names, but just what\u2019s the mix? What kind of people? On this committee.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">Three members were faculty members from Harvard Business School, and then the other four members were from within Harvard. So, there was a person who was from the Harvard Divinity School, the chair of the committee, and then there was one from the Harvard Graduate School of Education, one from the Law School, and one was a biology professor.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And these are all tenured professors.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">That\u2019s exactly right.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And I take it they\u2019re still carrying their teaching load at the time they\u2019re serving on this committee.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">Yes, I believe so.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so it\u2019s fair to say this is, in effect, a part time committee drafted by the University to weigh the evidence against you to determine whether HBS had met the burden that they had to meet in order to remove you. So what was that burden, what we call burden of proof that they had to meet?<\/p>\n<p class=\"para\">It\u2019s clear and convincing evidence of grave misconduct or neglect of duty.<\/p>\n<p class=\"para\">Okay, clear and convincing evidence. Not that there was some anomaly with the data that was involved with the papers, because that much was given, we all acknowledge that there was weirdness in the data. Rather, it\u2019s clear and convincing evidence that you had caused or intended to produce that anomaly, or those anomalies within the data, right?<\/p>\n<p class=\"para\">So this is not within your field of expertise. It\u2019s mine. Let\u2019s talk a little bit about what it means to say, clear and convincing. You know, ordinarily, in a civil lawsuit, the plaintiff or complainant or anybody who wants something from the court has to prove, prove something by what\u2019s called a preponderance of the evidence standard, and that just means that the Jury or the Judge, the Fact Finder must believe that the charge is more likely true than not. But that wasn\u2019t the standard in your case. In your case, the burden the business school had was not a preponderance of the evidence but clear and convincing, which is a much higher standard of proof. It\u2019s not quite the beyond the reasonable doubt standard of criminal law, but it\u2019s still very substantial.<\/p>\n<p class=\"para\">So now I want to just unpack this standard a bit, because when we get to the actual evidence, not with you today, but with others in later episodes, I\u2019m going to argue that the evidence doesn\u2019t even meet the preponderance standard, because your claim, and I\u2019m convinced of this as well, is not that you couldn\u2019t prove my guilt, but I\u2019m not guilty, period. I didn\u2019t do anything wrong.<\/p>\n<p class=\"para\">But the reason it\u2019s important to think about the standard here is because, again, it just makes clear how bizarre this whole action was, or bizarre that it was allowed to proceed as far as it did, because to meet the clear and convincing standard, the Fact Finder must have what\u2019s referred to as a &#8220;firm conviction of your guilt,&#8221; and that guilt must be &#8220;highly probable,&#8221; absent, &#8220;serious or substantial doubt.&#8221; So again, we\u2019re going to be asking, when we get to the evidence, whether that standard is met. And I\u2019m going to ask the listener when they hear the evidence whether they think that standard is met, but especially whether they think the standard is met in the way that you know, some prominent courts have described it. So the Ninth Circuit Court of Appeals said clear and convincing means, when the evidence, &#8220;leaves you with a firm belief that the conviction or conviction that it is highly probable that the factual contentions of the claim or defense are true.&#8221; New Jersey Supreme Court says the &#8220;evidence that produces in your minds a firm belief or conviction that the allegations sought to be proved are true.&#8221; Colorado, Supreme Court: &#8220;if considering all the evidence, you find it to be highly probable and you have no serious or substantial doubt.&#8221;<\/p>\n<p class=\"para\">I might be biased. Of course, I am biased. I have a view here. I\u2019m advancing my belief on the basis of that view, but I\u2019m going to predict that no fair listener, after they hear the evidence which will come in the later episodes, will be able to conclude that the evidence comes anywhere close to these standards. And not surprisingly, I guess you would say, because you have said, and again, I believe you are right about this, that you did not, in fact, commit academic fraud.<\/p>\n<p class=\"para\">Okay, so the Hearing Committee, the Faculty Committee that was to determine this, eventually was to receive these documents &#8211; the complaint, the answer, any exhibits associated with them, and then the testimony about them &#8211; and then, weighing that and the hearing, determine whether by &#8220;clear and convincing evidence,&#8221; they had established you were responsible for the data anomalies that had been identified.<\/p>\n<p class=\"para\">Now that was a big job. It\u2019s a hard job. So by the end, how many pages was the record that this committee was supposed to understand to do its work?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">Over 2,500 pages.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">2,500 pages. And during this process, beginning with the process of the Third Statute, what were the rules that were applied to you? Like, what were the obligations that you had during this process?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">The largest one was confidentiality. The Third Statute procedure stated that any public statement or publicity about the process had to be avoided. And then what\u2019s interesting is that the Hearing Committee, in a sense, made the rules around confidentiality even tougher. So in one of their memorandum to us, they indicated that this is &#8220;a confidential personal matter,&#8221; and so anything related to the Third Statute proceeding could not be disclosed. And in fact, if any information would to be disclosed, there would be consequences, because the Committee might determine that there was bad faith on the part of the person disclosing.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So you were essentially once again gagged in this process, as you were at the very beginning by the business school, watching the world talk about you and this process, but unable to talk back in this process. So when you think back on that, like, how did that make you feel as you watched that? Were you good at ignoring it or not paying attention?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">Absolutely not. I had so many emotions when learning about confidentiality, because it truly felt like I was in a cage. I finally had explanations for the anomalies. I was ready to talk about them, and I just couldn\u2019t publicize them. I just had to stay quiet. And what\u2019s interesting is that some of my co-authors reaching out were making comments about the fact that it seemed like I was just sitting and waiting for the leave to pass, and they had no idea about the process that I was going through, what it entailed and what the truth about the anomalies was.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So you believe you were developing a way to understand and explain what these anomalies were and how they were created. You couldn\u2019t talk about it, but everybody could talk about the anomalies on the outside, they could accuse you of committing fraud, knowingly committing fraud, and you\u2019re tied to not saying anything about it. Now, the weird thing about this confidentiality, to me, is that when I heard about it, I kind of thought it would be forever, but what was striking about their rules is that the confidentiality ended at the moment the process was over, right?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">Yes. But there was an additional confidentiality request before the hearing. There we were asked to sign an agreement, a confidentiality agreement, by which we couldn\u2019t talk about the hearing forever, and that is the point where I decided not to sign it. It just seemed really strange, especially in a world where HBS had pushed for the HBS report to be public, and now the world thinks that I have committed fraud when that was not the case.<\/p>\n<p class=\"para\">Right. So you would have been unable to do what you\u2019re doing right now: talk about this procedure. There was one side of the story that had been published and your side would not, and you, you and your lawyer said, No, you\u2019re not going to sign that. And at that point they backed off, right? And they said, Okay, well, you at least have to maintain confidentiality during this process.<\/p>\n<p class=\"para\">That\u2019s right. And it was a difficult decision, because, again, you\u2019re thinking through, \u2018will there be negative consequences by pushing back?,\u2019 but it felt the right decision to make.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Yeah, I think it was okay. So we have a process complaint, answer at a hearing. And so when was a complaint actually filed?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">I don\u2019t know the exact date, because when it was attached to the October 25 note, it was not dated, but probably late June, early July of 2023 okay.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And so this complaint incorporated the HBS report, right?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">Yeah.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So it\u2019s, it\u2019s based on the HBS report, yes. And so your job, then, or the job of your lawyers, was to file an answer to that complaint, and that\u2019s what you and your lawyers then worked on and you worked on it for one year, right? For one year, you were working through this complaint and building the response to the arguments they had made, to make it clear to the Committee that, in fact, the allegations were not correct. Is that right?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">Yeah, that\u2019s all correct.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So July 28 to August 1. And one of the most important elements of this complaint was a forensic report produced by a company called Maidstone, which had done the analysis to support HBS\u2019 conclusion that there was academic misconduct here.<\/p>\n<p class=\"para\">Okay, so let\u2019s talk about the answer that you finally completed after one year. One year worth of work. So how long was this answer?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">It was 93 pages, single spaced.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Wow. And were you proud of it? Were you proud of what it could show?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">I felt it was a really strong answer. In fact, I think I\u2019m very naive. I thought that when the answer was submitted, I thought that the Committee would look at it and read it carefully and possibly decide to have an open conversations about the fact that \u2018we recognize that HBS got it wrong, and so we\u2019re going to end the process.\u2019<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Yeah, I would say that was naive, but based on the answer, I can see why you could feel that, because it is an extremely powerful document, responding to the Maidstone claims, the complaint based on the Maidstone claims. And so when you think about what it showed, like, what do you think it actually showed?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">It did two things really well. First, it told this story for what it was. Why is it that we got to where we are? Why is it that Harvard HBS got it wrong? And importantly, it previewed the arguments that my independent experts would be making, and also it previewed my testimony, and it explained how Maidstone and HBS were wrong, not just about one of the allegations, but across all four.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so you\u2019re laying out exactly why the evidence that the business school had relied on was mistaken and that it didn\u2019t show what they thought it showed. And you had your own forensic experts who had looked at the same material to draw this conclusion, the conclusions they drew. And let\u2019s just emphasize again, because it was a long time ago when we did the other episode with you. You didn\u2019t get to have those forensic experts until after HBS had made its determination that you were guilty, they had the Maidstone report, and they determined you were guilty, and then you could hire your forensic experts, and your forensic experts could then point out the problems in the Maidstone report.<\/p>\n<p class=\"para\">Okay, now we\u2019re going to cover more in detail the specifics of the Maidstone errors in the subsequent episodes. But if you had to characterize the Maidstone report, what kind of mistakes did it make, and, more importantly, what kind of conclusions did it or did it not draw when it was looking at the evidence against you?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">What Maidstone did was to analyze different versions of data files for each of the allegations, and in particular, they were looking at an earlier version of a data set and a later version. And what they did is identified some discrepancies between the earlier version and the later version. But what they didn\u2019t do is explain the discrepancies. I don\u2019t even think that that\u2019s what they were asked to do. And also what they didn\u2019t do is to say these discrepancies are fraud and Francesca is responsible. That is actually a conclusion that the HBS investigation committee made, not Maidstone.<\/p>\n<p class=\"para\">And Maidstone also made some errors in their analysis. I think the one that stands out to me is in looking at the discrepancies they ignored the fact that there was a column in one of the datasets that said, exclude these participants. And so, of course, they saw a discrepancy, since the excluded participants were, in fact, excluded in the later data set. But then they realized that that was the case.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Yeah. So this is just a classic example of how had you had your own experts to review the Maidstone report at the time Maidstone made their report, before the business school committee had concluded you were guilty, they could have pointed out these mistakes. They could have pointed out the weaknesses and maybe Maidstone would have corrected them. Maybe they would have withdrawn them.<\/p>\n<p class=\"para\">But the point is, at that point, before a conclusion about your was made, there could have been an exchange to reveal the weakness of the foundation that this was all going to be based on. You have said that when you filed your answer, you had this naive belief that they would read it and say, \u2018huzzah. Okay, we\u2019re finished. We see that there\u2019s no guilt here, or at least no ability to show, by clear and convincing evidence, that you\u2019re guilty.\u2019<\/p>\n<p class=\"para\">You filed this on August 1, 2024 and two weeks later, ish on August 16, Harvard was supposed to submit its testimony in support of its complaint. To be clear, it\u2019s not the complaint, but the evidence to support the charges in the complaint is that, right?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">Yeah, that\u2019s my understanding.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And I expect you were expecting that part of that testimony would again be the Maidstone reports.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">Yeah. And throughout the Third Statute proceedings, HBS kept saying that the HBS investigation process was a &#8220;painstaking and comprehensive process,&#8221; and they kept referring to the report as the reason for the Third Statute proceedings. So what I expected to do was to just respond to those analyses.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">In fact, that\u2019s what you had spent the last year working on responding to.<\/p>\n<p class=\"para\">Okay, so then August 16 happens, and Harvard makes its filing, and its filing was not quite what you expected it to be, right. So what did Harvard do on August 16?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">HBS dropped Maidstone, and what they submitted instead was a 230 pages report by Stanford Professor Jeremy Freese.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so who\u2019s Jeremy Freese?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">Jeremy Freese is a professor of sociology, and what he also does he is interested in social science methods.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so he is a tenured professor at Stanford. Does he typically do kind of expert reports like this?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">No, and in fact, he mentioned that this was the first opportunity that he had to do something like this.<\/p>\n<p class=\"para\">Okay, so you\u2019re expecting evidence that\u2019s explaining the Maidstone report and the arguments from that to demonstrate your guilt, you had spent a year preparing the response to that. This 230-page report is dropped on you, and the Maidstone report is gone, and this becomes the basis of the Harvard prosecution against you, this new report.<\/p>\n<p class=\"para\">Yes, and I remember reading it, and it was a shock. I felt shattered. I remember standing in front of my husband, Greg, and trying to explain the situation, the disbelief, and asking him to take the kids away so that I could spend the next month just trying to address this additional report. And it just felt awful. I had sent my kids away with my husband earlier to focus on the response, and now I was doing again the same thing. And I remember my husband looking at me and say, &#8220;You\u2019re in the ninth inning. Stand up straight and give it your best game. I\u2019m going to take off with the kids and let you be.&#8221; And I just had so many tears coming on my face.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Yeah. I mean, it\u2019s literally astonishing. I mean, it\u2019s not astonishing that they would want to do this, given the weakness in the Maidstone report, the reports that HBS had relied upon to find your guilt in the first place. It\u2019s not surprising they would want a different report, but I think it\u2019s literally astonishing that this would be allowed because you had spent, how much money had you spent at that point to write the defense to the Maidstone report in the original complaint over the course of that year?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">It was over $2 million.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">$2,000,000. 14 months of work. A year plus the time before that. And now you had to start over with a brand new report. But even worse, you had had one year to prepare the response to the first Maidstone report, and now you were given one month to prepare a response to the second. And was that all you had to do in the course of that one month before everything was supposed to be completed?<\/p>\n<p class=\"para\">Not really. I had to respond to the report that Freese wrote, but that also required going back to the data, trying to understand his analysis, and point out the places where he was wrong. And I had to figure out how to do that in a way that didn\u2019t create a Frankensteintype response, because I had the Data Colada allegations to address, the Maidstone reports to address, what the Committee suggested, and it was almost starting from scratch. And I remember being home alone, and every day of that month, I woke up and glued myself to my chair and worked for 14-16 hours a day. And you have to also remember the context. We have lawyers on each side, and as we are trying to pivot and respond to the report, there are constant back and forth in between lawyers for requests that Harvard is making. At that point, he was the forensic image of my computer.<\/p>\n<p class=\"para\">Okay, so let\u2019s focus a little bit about how the Freese report was actually different from the Maidstone report. Like, what was it? If you had to summarize, characterized the big differences between the two reports.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">It had new charges, he brought in new evidence to sustain those charges, and he also presented very different analyses. But what was also striking is that Freese did something that Maidstone did not do. He constructed what he called &#8220;falsification scenarios&#8221; that, according to him, explained how the anomalies could have been created in a way that indicated fraud.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Falsification scenarios.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">And personally, I found it really hard to read what he wrote, because he had such great confidence in these falsification scenarios, and yet they came from a person who knew nothing about my research practices and how I worked.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So these were hypothetical ways in which fraud might have been committed. And the question was whether these would convert into &#8220;clear and convincing evidence,&#8221; evidence that is firm and creates, in the minds of the Fact Finder, a firm belief or conviction that it is highly probable that they are correct. That\u2019s the aim that they were trying to achieve.<\/p>\n<p class=\"para\">And this falsification scenarios was the first time&#8230; I just want to emphasize this point, because it\u2019s important, the first time in the whole of this process where people had painted these hypothetical scenarios as a way to produce evidence that this is, in fact, what you had done, right?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">That\u2019s right. And the report came at a really tricky moment for some of the analyses. Again, I wanted to have more information. And by then, discovery was over.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Yeah, let\u2019s be clear about that. When he produces new charges and new evidence, if you\u2019re going to respond to that, you might yourself need some evidence. So the typical way you would do that is ask for discovery, the process of getting evidence from the other side. But the timing for discovery was over. The rules of the committee forbid you from having any more discovery, right?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">That\u2019s exactly right. No more discovery.<\/p>\n<p class=\"para\">Lawrence Lessig So you have this one month period where you couldn\u2019t get even the evidence to respond, to the extent you needed the evidence to respond to these new charges, new allegations and these falsification scenarios. You know, Stanford\u2019s not far from Hollywood, but maybe this is an aspiration to his Hollywood career as he writes the theory of how fraud is conducted.<\/p>\n<p class=\"para\">It just felt like I had got punched again and I had my hands tied behind my back.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Yeah. Okay, so you had great lawyers. I really liked your lawyers. They objected to this new report. They objected to this whole process, right? They asked that the report not be considered. Stated that if the Hearing Committee were to endorse the substitution of this new report over the Maidstone report, &#8220;the integrity of the hearing will be compromised and will be conducted under protest and over the strong objection of the respondent,&#8221; you, right?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">Yeah.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And were they surprised themselves? I don\u2019t want to breach confidence here, but was this something that anybody was expecting?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">Everybody was incredibly shocked. And again, not only I had to pivot, but all my experts had to pivot.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Yeah, pivot because they had developed expert reports based on the allegations that had originally been made. Now there are a whole bunch of new allegations that were made, and so they needed to shift what they were looking at and how they were looking at it. That\u2019s right. And again over a very short period of time.<\/p>\n<p class=\"para\">Okay, so your lawyers objected. The objection was denied. You asked for more time.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">I did ask for more time, since we needed it.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And you didn\u2019t get any more time.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">We did not.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So let\u2019s be clear about the core mistake here, right? You know, the Hearing Committee was free to consider whatever evidence it wanted. But HBS had made it clear that it didn\u2019t really consider the Hearing Committee to be what we would call a primary Fact Finder. I mean, that\u2019s lawyer speak for you know what the, for example, ordinary jury trial court does? They hear the evidence, they weigh the disputes, they make a determination.<\/p>\n<p class=\"para\">It was clear that Harvard Business School didn\u2019t want the Hearing Committee to do that, because when you had asked for the time to lay out all of the evidence, and your response to the evidence you\u2019d asked for six days, the business school said you don\u2019t need six days. You only needed two days because all of the factual work had already been done by the Investigative Committee in the business school. So now we\u2019ve already gone through the mistakes that that committee made originally &#8211; the process mistakes, gagging you and forbidding you from actually investigating the charges against you, forbidding you from attaining your own forensics firm. And of course, that made this Investigative Committee a particularly terrible fact-finding court. No court gags the defendant or denies her the freedom to develop expert support to defend herself.<\/p>\n<p class=\"para\">But now the point is even stronger because Harvard itself had determined that what the Investigative Committee had done was flawed because it was based on a report that the business school was no longer using to prosecute the case against you. It decided to withdraw the Maidstone report, substitute a whole new report, and that new report was really new. The evidence was new. The charges were new, the theories, the falsification scenarios, were new.<\/p>\n<p class=\"para\">And so given this fundamental shift, I would have thought, you know, the obvious thing for the Hearing Committee to do was to say: hey, hold up here. You know, in the language of judicial litigation, you could have said: Look, you don\u2019t get the opportunity to retry your case in this appellate court. You either defend the judgment or we\u2019ll send it back. You can try it again in your in your Investigative Committee, and see whether what Freese says stands up. But it\u2019s completely wrong for what in effect the business school had argued should be an appellate court to basically hold its own trial. Or if that\u2019s what they\u2019re going to do, it\u2019s completely wrong to do that while limiting your opportunity both to develop a defense by giving you just one month to respond to a 230-page report, and by presenting limit on the defense, they gave you just two days to now respond to all this new evidence.<\/p>\n<p class=\"para\">So I\u2019m going to call this fundamental mistake number two. The first fundamental mistake was gagging you during the investigation. The second now is for the Hearing Committee to become, in effect, the primary Fact Finder. And not just because this is not what a part-time committee could reasonably have been asked to do, but also because it conducted this primary fact finding enterprise so incredibly poorly, and even now worse, you\u2019re faced with a record that\u2019s just a total mess.<\/p>\n<p class=\"para\">So what did your team at this point believe they needed to accomplish in their final filings with the Hearing Committee?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">I think you said it correctly and well: it felt like a mess, because this fundamental mistake had a really important implication, which was, we needed to defend what\u2019s in the record. We needed to build on the response that previewed our arguments. But in addition to that, we needed to really dig deep into the 230-page report by Freese and ensure that every single explanation that he provides or different theories, we address it.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Yeah, so you\u2019re fighting on four fronts at the same time, including the Data Colada allegations, so all of these things are out there for you in this short time to respond.<\/p>\n<p class=\"para\">So by September 13, less than a month after they replied, you needed to answer everything in this new complaint, which you did. What exactly did your team then produce in response to all of this stuff?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">So my testimony, my reply, was 181 pages. My forensic experts submitted a report that was 203 pages long. Then I had data experts that submitted a 43-page report, and a behavioral science expert that submitted a report of 64 pages. And then we had written testimony by three different witnesses.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so in the end, Harvard had submitted about 1700 pages against you, considering the report, the complaints, the various experts and fact witnesses, plus the supporting materials, like the data files and copies of the relevant emails you\u2019d responded with, about 600 pages. So that\u2019s a total of about 2300 pages for this part time faculty committee to work through without considering the supporting materials that they could also be referencing.<\/p>\n<p class=\"para\">Okay, so two weeks after that, two weeks after September 13, the end of September, Harvard was given the last word against you. What did they file at that point?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">At that point, Dean Datar and the research integrity officer submitted a supplemental statement, and then Freese submitted 112-page rebuttal to our September 13 submission. And then a forensic expert that was hired by HBS submitted a 62-page rebuttal to<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so all of this is weird, because, you know, normally, at least in a criminal proceeding, it\u2019s the defendant that gets the last word on the facts. So here they are giving the last twist to the facts that the committee is going to have a chance to review. But okay, you know, it\u2019s not a criminal trial. Maybe, maybe just say, whatever about that.<\/p>\n<p class=\"para\">But the point is, there\u2019s a big mess sitting before the Committee, and so the Committee now is to conduct hearings, and there are just two days of hearings scheduled, and this is November 15 and 16th, which I think are Friday and Saturday.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">That\u2019s right\u2026<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">\u2026because that\u2019s the only time they could fit, because they all are busy professors doing all the things busy professors do. And so the hearing begins. Fill in the context a little bit\u2026 what is the hearing like? Like, where is it? What\u2019s the room feel like?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">I think that people might feel like it\u2019s a science presentation, and instead, it was not. The room felt very much like, and looked very much like, a courtroom. So you have the Committee sitting on one side of the room with their counsel by their side, and also a chair where people, in a sense, would get deposed or cross examined. There is a podium in the middle of the room, and then behind the podium you have one side sitting with the lawyers, and the other side sitting with the lawyers. And I remember not even being able to see some of the committee members, which in a context like this one, was really hard\u2026<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">\u2026they were far away, or because?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">Because the podium was right in the middle of it, and it\u2019s blocking the view. And so I was there with my forensic expert, and HBS had their forensic experts in the room, and then the witnesses would come in at the time at which they were cross examined. And it was just a surreal moment and a surreal room. HBS brought these large billboards to show the Committee, and that was really strange. And if you just looked around, there were more lawyers than faculty members.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So HBS has one set of lawyers from, I think Ropes &amp; Gray is their lawyers, right?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">That\u2019s right.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Really great Boston firm, really, really good lawyers. And then the Committee, you said, had its own lawyers, or lawyer, I guess, was one lawyer was present. Were there more present?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">I believe there were a couple of them, plus Harvard General Counsel was present. So I think there were five or maybe even six lawyers on the HBS side, I had two. There are lawyers everywhere, yeah.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So you had two, and then then the committee had two or three, and then there\u2019s just seven members of the committee and you, and then the witnesses.<\/p>\n<p class=\"para\">Okay, so you were given six hours of testimony, or six hours to present your side of the case. And so in those six hours, you were supposed to examine the HBS witnesses, present your own defense, show the mistakes in the Maidstone report, which, though Harvard had walked away from it, the Committee was free to consider it if it wanted show the mistakes that HBS had made based on the Maidstone report, show the mistakes in the Freese reports, and also try to explain your theory of what had happened with these four papers written up to 14 years before, as we\u2019re going to get to in a minute to explain how these anomalies had been produced.<\/p>\n<p class=\"para\">So when you when you think back on the hearing, leaving the hearing after those two days, how did you feel about what had been presented?<\/p>\n<p class=\"para\">The hearing was very contorted, in a sense. Again, rather than talking science, you have time that is constrained, where you have the opportunity for direct arguments, and then you\u2019re cross examined by lawyers. So the feeling was weird as I left the room, but I remember thinking that if the Committee decided on the merits of the case, then I would be back to HBS as a professor. I felt good about the arguments that were made.<\/p>\n<p class=\"para\">And then I remember feeling really exhausted. I was cross examined on day two of the hearing, and I basically woke up with no voice whatsoever, which somehow felt very fitting to the moment. Since, again, he was just a really constrained process.<\/p>\n<p class=\"para\">Were there big surprises that came out of the hearing that you hadn\u2019t expected?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">The most shocking one was the Research Integrity Officer when he was giving his testimony. There were a lot of things that he said that either he has a bad memory or they were just lies. So he, for example, claimed that he encouraged me multiple times to hire a forensic firm. That\u2019s not true, but when he was asked about those conversations, he said, \u2018I don\u2019t remember.\u2019 Or he mentioned that the evidence in the HBS investigation was not full evidence, because I somehow directed which files to copy on my computer. And when asked why is it that Francesca directed which files to copy, he said, \u2018Oh, because we wanted to protect her privacy.\u2019 And as it turns out, he was so concerned about my privacy that there are 400,000 files in a folder called personal that has family photos and my tax returns. And so it was just painful and shocking to hear him say, consistently, \u2018I don\u2019t remember.\u2019 And again, this is the Research Integrity Officer.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">He didn\u2019t remember, except he did remember the critical fact which would negate the strongest charge against the Business School Committee, which was that they gagged you and forbid you from hiring a forensic firm when they themselves had hired a forensics firm. Yeah, that must have been hard to watch.<\/p>\n<p class=\"para\">Okay, so after the hearing, three weeks later, you guys have to submit another post-hearing document, filing some 20 pages. And then seven weeks after that, on January 24 the Hearing Committee issued its final findings and recommendations, and those findings were against you, and the Hearing Committee recommended that your tenure be revoked.<\/p>\n<p class=\"para\">Now let\u2019s talk a bit about this document that they released. I\u2019m going to post it on the website and on the Substack. We know from the header data that it was drafted by the lawyers working for the Committee. Might not be surprising, I don\u2019t know but, but you know, you\u2019ve got a 2500-page record. How many pages long was this document with findings?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">It was 11 pages.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, and in those 11 pages, how many citations are there to the record that had been developed?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">There was none. It was actually quite shocking not to see any.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Yeah. I mean, you know, even the United States Supreme Court feels obligated to point to the evidence that supports its findings or its claims.<\/p>\n<p class=\"para\">This document reads like edicts from Zeus, but as we\u2019ll see in the episodes that follow, that this was not the work of Zeus, or at least an omniscient Zeus, because we\u2019ll see the obvious mistakes and flat out falsities that are within this report, we\u2019ll see it does nothing to explain how it could conclude by clear and convincing evidence that what it did, established your guilt.<\/p>\n<p class=\"para\">But the point I want to emphasize here again is just the failure of this process. They had air dropped new charges three months before the hearing. We\u2019ll consider the rebuttal. Considering the rebuttal, you could say it\u2019s one and a half months before the hearing, but with no real, meaningful opportunity to respond.<\/p>\n<p class=\"para\">The record was already complex, extremely long, and these were already part time fact finders.<\/p>\n<p class=\"para\">We\u2019ll work through not with you, but with others, what their mistakes were.<\/p>\n<p class=\"para\">But at some point it should have been obvious that there was something deeply flawed about this whole enterprise.<\/p>\n<p class=\"para\">Harvard had spent many more times than you could afford to buttress the conclusions of an HBS Faculty Committee, conclusions based on a report that HBS itself abandoned, but rather than rerunning that process after they had abandoned that report, rather than a new HBS Committee being given a chance to evaluate the charges against you grounded in Freese\u2019s report and give a real chance to respond to Freese\u2019s report, Harvard used the Hearing Committee essentially as this Primary Fact Finder, and that fact finding had gone against you.<\/p>\n<p class=\"para\">And so at this point, when I read this, I volunteered to try to carry the burden of the next step in this case.<\/p>\n<p class=\"para\">I mean, I thought your lawyers had been great, but I was eager to help, because as I unpacked what had happened, it struck me that this case was not really just about you. It was, of course, first about you, the extraordinary injustice that you had suffered. But it was also about this astonishingly bad process by Harvard, not just the Business School process that had gagged you when you needed to actually mount a defense, but this whole bait and switch process with the Third Statute proceeding. Yes, spend a year and $3 million building a defense against this complaint, and then, oops, sorry, you need to race in the next month to build a new defense against a new complaint with multiple new charges and new theories and falsification scenarios added in, like icing on a cake.<\/p>\n<p class=\"para\">And so when I thought about that, it just felt like insult added to injury when I read this extraordinary line from the hearing committee\u2019s report, and I\u2019m going to quote it in its full quote:<\/p>\n<p class=\"para\">Much of Professor Gino\u2019s presentation at the hearing focused on her criticisms of the HBS inquiry and investigation. Although we do not find her criticisms compelling, they are ultimately irrelevant to our determination because we did not defer to or rely on the outcome of the HBS inquiry or investigation, nor did we limit this proceeding to arguments or evidence presented in that forum.<\/p>\n<p class=\"para\">Now I don\u2019t know. Maybe I\u2019m too naive, too idealistic in my understanding of what fair process would be, but anyone who would look at this process &#8211; from the gagging of the defendant during the most critical time to gather evidence, to the fact that the report that was the basis of the finding of guilt was essentially withdrawn, to the bait and switch on the charges being made against her, to a forensic report filled with charges and evidence never reviewed by anyone &#8211; anyone who would look at that process and say that the &#8220;criticisms&#8221; were not &#8220;compelling&#8221; obviously, in my view, this is a technical legal term, does not know jack shit about process. This process was an embarrassment.<\/p>\n<p class=\"para\">No one accused of academic misconduct should be restricted in developing a defense before a committee concludes she is guilty.<\/p>\n<p class=\"para\">In developing that defense, the target must be free to hire whatever experts may help her make her case to the initial fact-finder.<\/p>\n<p class=\"para\">She must be free to interview anyone who would have relevant knowledge to demonstrate her innocence.<\/p>\n<p class=\"para\">She must be given adequate time to develop her defense before a committee determines her guilt.<\/p>\n<p class=\"para\">Should an allegation make it as far as Third Statute proceedings, the allegation must be fixed at the start of such proceedings, not subject to changing claims and new experts arriving midway through.<\/p>\n<p class=\"para\">And most obviously, if it does change, the Hearing Committee should just return the process to the school raising the complaint. Do your own hearing, and bring those results to the Hearing Committee. The idea that the Hearing Committee should be a fact finder \u2014 as they said, &#8220;we did not defer to or rely on the outcome of the HBS inquiry or investigation, nor did we limit this proceeding to the arguments or evidence presented in that forum&#8221; \u2014 is flat out bonkers.<\/p>\n<p class=\"para\">And anyone who doesn\u2019t see its embarrassment is simply not looking. Okay. So Am I overreacting here? Francesca, or is that how it felt to you?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">It felt that way for sure. It felt as if the target kept moving.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Yeah, and that\u2019s a problem with process. With fair process, and there was no fair process here.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">It felt very unjust.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Which brings us finally to one more extraordinary error of process. So as we\u2019ll be clearer when we get to the four allegations, three of those allegations were about papers written a long time ago, when were the allegations related to two through four written?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">Allegation 4 was about a paper now 13 years old. Allegation 3 was about a paper now 11 years old. And allegation 2 was about a paper that is now 10 years old.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so you\u2019ve had, you\u2019re not terribly old. You\u2019ve had a significant career. So you know, how many years have you been in this business?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">Over 20\u2026<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, 20-25 years. You\u2019ve written how many papers?<\/p>\n<p class=\"para\">Okay, so we\u2019re now talking about four papers over these 25 years, and the allegations in this case are about four papers, three of which are at least 10 years old. Now anyone with any sense of fair process, would ask a pretty obvious question here. Isn\u2019t there the equivalent of a statute of limitations about such charge? Is it really the case that an academic can be forced to defend a 13-year-old paper, forced to show how the data for that paper was collected by whom and when? For example, as you hear in the next episode, allegation 4 was about a study conducted in 2010 while you were at the University of North Carolina, right?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">Yeah\u2026<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And so tell us about that study.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">That was a study where my coauthors and I were looking at whether signing a pledge of honesty would affect the honesty in the reporting by the person who signed. And so we had different conditions, three conditions, one in which there was no pledge of honesty, one in which there was first the pledge of honesty and then the reporting of what participants did. And then there was a condition where first participant reported their performance and then they signed a pledge of honesty. And the idea was that people would be more honest if they signed the pledge of honesty first.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so how was the data collected for that study, wWay back in 2010?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">It was collected on paper, which was very common at the time.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And so, as it just turns out, and amazingly, makes you seem a little bit like a pack rat, but amazingly, you discover that you still had the paper receipts for this, right?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">That\u2019s right, and it\u2019s not so surprising, because when this study was conducted, we\u2019re back in July of 2010 I was in between jobs. So I was moving from UNC to the job at Harvard that would officially start on August 1. So I likely brought the receipts with me.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so you packed them in a box, and they put it in the moving van, and it was moved up here and thrown into your garage, or something like that. So it turned out to be hidden in your garage. And so, you know, you might ask, \u2018Well, what if you didn\u2019t happen to have the receipts?\u2019 Because these receipts as well here, when we go through this episode, are really important in establishing your innocence with respect to the charge that was made here. But you know, what if you hadn\u2019t had the receipts? You know, the fair process question is, \u2018Can it really be fair, right, that somebody is forced to defend something a decade ago, that they would have no reason to continue to have the evidence to be able to defend yourself?\u2019<\/p>\n<p class=\"para\">Now some people might hear that and say, why not? You know, fraud is fraud. Why should we allow somebody to get out get away with fraud?<\/p>\n<p class=\"para\">But the reason we have statutes of limitations is not to allow someone to get away with something. It is instead because we understand that evidence grows stale\u2014and it doesn\u2019t necessarily grow stale in an even way. It might grow more stale for the defense, as it were, than the prosecution. Either way, it becomes incomplete. And any effort to understand what happened a decade ago will be skewed by the incompleteness of the evidence. No one keeps every email. Harvard by default flushes email after a number of years. Harvard\u2019s own data retention policy (for research materials) is 7 years. But maybe one party in a dispute is obsessive about archiving his own email. Then the facts will be framed around what he saved, even if, what he saved is a skewed and incomplete picture of what actually happened.<\/p>\n<p class=\"para\">All this is why, in fact, there is a statute of limitations for charges of academic misconduct. As the rule provides:<\/p>\n<p class=\"para\">&#8220;allegation about research that is more than 6 years old cannot be investigated, unless&#8221;: the [scholar] has continued or renewed an incident of alleged research misconduct through the citation, republication, or other use for the potential benefit of the respondent of the research record in question.<\/p>\n<p class=\"para\">Okay, obviously the &#8220;unless&#8221; clause here is pretty complicated.<\/p>\n<p class=\"para\">Let\u2019s do it again\u2026 &#8220;UNLESS the [scholar] has continued or renewed an incident of alleged research misconduct through the citation, republication, or other use for the potential benefit of the respondent of the research record in question.&#8221;<\/p>\n<p class=\"para\">Okay, let\u2019s do a little bit more to unpack this mess.<\/p>\n<p class=\"para\">It\u2019s clear that the &#8220;unless&#8221; clause is saying that staleness is not an excuse if you have &#8220;continued or renewed an incident of alleged research misconduct.&#8221;<\/p>\n<p class=\"para\">That much makes sense. If you have a dataset that you fabricated in 2000, but use it again in 2025 to write another paper consistent with its fabrication, the mere fact that you had first used it in 2000 can\u2019t block you from being charged with fraud in 2025.<\/p>\n<p class=\"para\">The ambiguity comes from the specification of how one &#8220;continues or renews an incident of alleged research misconduct.&#8221;<\/p>\n<p class=\"para\">The language after that clause states &#8220;through the citation, republication, or other use for the potential benefit of the respondent of the research record in question.&#8221;<\/p>\n<p class=\"para\">So that complicated clause has three parts. One is &#8220;through the citation, republication, or other use&#8221; Second part is &#8220;for the potential benefit of the respondent&#8221;<\/p>\n<p class=\"para\">Third part is &#8220;of the research record in question.&#8221;<\/p>\n<p class=\"para\">So that too is a mess.<\/p>\n<p class=\"para\">One simple way to make clearer what it\u2019s trying to say is to reorder the clauses a bit.<\/p>\n<p class=\"para\">So, for instance, &#8220;through the citation, republication, or other use&#8221; &#8220;of the research record in question.&#8221;<\/p>\n<p class=\"para\">&#8220;for the potential benefit of the respondent&#8221; But even this clarification raises an obvious question: If I simply cite an article I wrote 20 years ago, is that my &#8220;continuing or renewing an incident of alleged research misconduct through the citation, republication, or other use of the research record in question for the potential benefit of the respondent&#8221;?<\/p>\n<p class=\"para\">In a literal sense, it seems the answer to that question must be yes. Why am I citing the article unless to benefit myself, and the rule expressly mentions &#8220;citation.&#8221;<\/p>\n<p class=\"para\">But the problem with literal interpretations is that they are often literally absurd. Because what this interpretation means is that if an academic keeps a list of all his citations on his website, or cites them in a string cite in a paper, he is persistently liable for a charge of academic misconduct for everything he\u2019s written for as long as he\u2019s been writing. Or put another way, the only way, as an academic, to get the benefit of the statute of limitations is to stop citing an article after it is 6 years old.<\/p>\n<p class=\"para\">The drafters of this rule realized this problem. In September 2024, they clarified the rule, specifying that &#8220;the subsequent use exception applies to the &#8220;citation to the portion(s) of the research record \u2026 alleged to have been fabricated \u2026 for the potential benefit of the respondent.&#8221;<\/p>\n<p class=\"para\">Now, in my view, this is a completely sensible interpretation. It\u2019s a little bit opaque. We could unpack it like this. What it is saying is that the &#8220;subsequent use exception&#8221; \u2014 the exception that allows them to prosecute a 13 year old paper \u2014 depends on showing that the academic has cited &#8220;to the portion of the research record alleged to have been fabricated.&#8221;<\/p>\n<p class=\"para\">So if you cite a paper that pulls together a bunch of research but not point to the &#8220;portion of the research record alleged to have been fabricated,&#8221; that\u2019s not subject to the exception. But if you pull the alleged fabrication out and write a new paper based on it, it might be.<\/p>\n<p class=\"para\">Okay, so that complicated rule, but again, it\u2019s their job to interpret the rule that they\u2019re supposed to be implying that complicated rule. How do you see that complicated rule applying to the four allegations?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">In this case, when I use the clarification to the rule, then what you see is that I did not cite the portion of the research record that was said to have been fabricated within the last six years.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So what you did, and what we\u2019re going to do is we\u2019re going to post all of the examples of your citing so people can look at how you cited them, and you can see that how you cited them is basically the way that you would cite just a string side of things that you had worked on in before. You would not cite to the portion of the research record said to be fabricated. And so if that\u2019s true, you would not be within the exception, which means that if that\u2019s true, you should not have been prosecuted for all four of these papers. At most you should have been prosecuted for just one.<\/p>\n<p class=\"para\">Okay. Now, astonishingly, the HBS Committee didn\u2019t even bother to explain why it was allowed to investigate because, remember, the rule says you cannot investigate the charge if it\u2019s more than six years old. They didn\u2019t even explain why they were allowed to investigate these three papers despite the rules.<\/p>\n<p class=\"para\">The Third Statute proceeding did explain why they were allowed to investigate and prosecute for these despite the rule, but it held that the mere citation of the article, not the citation or use of the allegedly fabricated data was enough to trigger the exception.<\/p>\n<p class=\"para\">And my view is that is just a plain misreading of this rule.<\/p>\n<p class=\"para\">And again, you listeners can verify. You can look at the citations as we post them on the website. You decide whether it fits this exception, which we\u2019ll also post with a little bracket that shows you exactly how it has changed.<\/p>\n<p class=\"para\">But as I read this reinterpreted rule, what this means is that three quarters of this case should never have been in the case, three quarters just disappear, which means that 75% of the over $3 million that you have spent, and 75% of the who knows how much Harvard has spent? Obviously, much more than you, because they had five times the number of lawyers you had working on this case would never have been spent.<\/p>\n<p class=\"para\">Now again, this argument doesn\u2019t wipe away all 4 allegations. One would remain. And in the next episodes, as we address the evidence for each, we\u2019ll see that the evidence for this one is also embarrassingly weak. But I want to call this fundamental mistake number 3: And recognizes just how astonishing is the waste that this process has spread, given the narrowness of the only legitimate case that could have been brought here. Data Colada was free to raise questions about whatever it wanted. It wasn\u2019t bound by a rule that said that allegations older than 6 years &#8220;cannot be investigated.&#8221; But Harvard was bound by that rule. Despite being forbidden\u2014under its own rules \u2014 from &#8220;investigating&#8221; three of the 4 allegations against you, it has forced you to defend all four.<\/p>\n<p class=\"para\">Maybe someone would say, Harvard wasn\u2019t forbidden. It was free, someone might say, to interpret the rule in the absurd way \u2014 in a way that means no one is ever free from an allegation of misconduct.<\/p>\n<p class=\"para\">True. But then at the very least, Harvard should be honest about the fact, and not pretend, either to the public, and certainly not to its faculty, that it gives anyone the benefit of a limitations rule. Under this interpretation, it does not. Under its interpretation, citing work you have done in the past is enough to expose you to the burden of defending yourself against a charge of fraud from that work, however old.<\/p>\n<p class=\"para\">And that burden, as we\u2019ve seen in Francesca\u2019s case, is an extraordinary burden.<\/p>\n<p class=\"para\">Okay, so let\u2019s bring this episode to ground. You submitted your appeal of the Hearing Committee\u2019s determination on March 14. When was the next you heard from the administration?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">It was May 20 in 2025<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And how did you hear from them?<\/p>\n<p class=\"para\">Through a one paragraph email.<\/p>\n<p class=\"para\">So we\u2019ve got the email here. I want to give it to you and ask you to read it. Can you read the email?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">The subject is &#8220;Message from President and Fellows.&#8221; And I\u2019m going to read the text:<\/p>\n<p class=\"para\">At the request of the President and Fellows of Harvard College, I write to inform you of the President and Fellows\u2019 decision to accept the findings and recommendations of the hearing committee empaneled to consider the matter concerning Professor Francesco Gino of Harvard Business School pursuant to the Third Statute of the University, and to remove Professor Gino from her appointment as a tenured professor at Harvard University. And then there is the signature.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so when the Dean of the Business School informed you of the Business School\u2019s decision, they had the character at least, to call you into the office and confront you face to face. Here\u2019s Harvard dismissing you effectively in a one paragraph email. Is there something you noticed about that email?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">They misspelled my name. It\u2019s Francesco instead of Francesca.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So they send you an email that misspells your name to tell you and then dismissed. They haven\u2019t responded to your appeal. They\u2019ve given you no explanation beyond that. And so when you read that, notice this, how did that make you feel?<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">It felt cruel. It lacked humanity. I had spent 15 years giving my heart and mind to the institution. I taught, I did research, I mentored and advised. I thought of myself as a good citizen. I am certainly far from perfect and if I were to go back, I would spend time thinking about how to improve the practices in the field. But I did not commit academic misconduct.<\/p>\n<p class=\"para\">And that email was such a hard reality. I had spent almost two years on unpaid leave, navigating through a process that was truly an insane emotional roller coaster. And here we are, about three weeks before the end of my leave, and I got fired this way.<\/p>\n<p class=\"para\">And I kept thinking about what this university stands for, which is Veritas, and with this motto in mind, I just still\u2026 I didn\u2019t understand what I went through. And to this day, I don\u2019t get it. It was not a search for truth. It was just a really painful process that I don\u2019t wish on anyone.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">There will be some who hear this and think to themselves, okay, fine, but if you committed academic fraud, you got what you deserved, and I think it\u2019s a fair thought.<\/p>\n<p class=\"para\">In the next episodes we\u2019re going to work through whether anybody could fairly believe that the evidence shows that you committed academic fraud. And so from my perspective, it\u2019s both because I think there\u2019s no sufficient evidence that you committed academic fraud, and that I think that this process was an embarrassment to a great university, that I thought it is important that we find a way to tell this story.<\/p>\n<p class=\"para\">So I\u2019m grateful you\u2019ve put yourself through these two interviews. These will probably be the last times we talk, although, depending on how the next episodes unpack, we might find it helpful to bring you back at the end, but I\u2019m grateful you would suffer this for us too.<\/p>\n<div class=\"speaker-tag sp-francesca\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Francesca<\/div>\n<p class=\"para\">I really appreciate you talking to me, Larry, and I appreciate the courage that you\u2019re showing and making sure that people know more about my story, my side of the story. But also, I hope that it causes people to pause and reflect on the type of processes that were used here and that that changes.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">That ends the third episode of this season of the podcast, &#8220;The Law, such as it is.&#8221;<\/p>\n<p class=\"para\">In the next episodes, we will turn to the actual papers and the claims of academic misconduct made against each of them. I will be joined in that conversation with a friend who has studied this case as carefully as anyone.<\/p>\n<p class=\"para\">And as I said at the start, I have invited Data Colada to participate in these conversations, but I\u2019ve not yet received any reply to my request. Now, some might think it weird to ask Data Colada to participate. They, after all, began all this, but as I\u2019ve said from the very beginning, I don\u2019t fault them for raising questions. I praise them for raising questions. Once they had raised those questions, it was for Harvard to fairly and accurately evaluate whether the anomalies that Data Colada had identified could fairly be tied to Francesca\u2019s intentional actions. They didn\u2019t conduct that investigation. They\u2019re not responsible for whether Harvard got it right or not.<\/p>\n<p class=\"para\">So I was hopeful, I remain hopeful, that they could participate in the post-mortem of what Harvard did and what Harvard concluded, to see whether they agree with the decisions that Harvard has made based on the evidence that was adduced beyond the anomalies they had identified. We\u2019ll see. Stay tuned.<\/p>\n<p class=\"para\">This podcast, &#8220;The Law, such as it is,&#8221; is my production. The actual producing of it is done by Josh Elstro of Elstro Productions.<\/p>\n<p class=\"para\">This episode marks the end of the conversation with Francesca, and the next two or three conversations will then cover the substance of the charges, as I\u2019ve just said. And then at the end, we will have a reflection on what we might learn from the whole of this extraordinary process.<\/p>\n<p class=\"para\">I\u2019ve been inspired or condemned, you can decide which, to try to tell this story through a podcast, in part because there are so many different parts to think through. And to think through them slowly is, I think, the best way to think through this issue.<\/p>\n<p class=\"para\">But it\u2019s hard for some to keep up with the pace or to slow down to the pace. And the classic picture, perhaps this is the best image of the moment we are in, in our time as a culture.<\/p>\n<p class=\"para\">Someone tweeted about this new podcast. Here\u2019s what they said \u2014 remembering, of course, that from the very beginning, I\u2019ve described how each step of this podcast will introduce more of the case, and that the final proof that there is no evidence to convict Francesca will come at the end. After the second episode was released, this is what this tweeter said,<\/p>\n<p class=\"para\">&#8220;a new passionate defense of Francesca Gino, this time a podcast series by Lawrence Lessig. I\u2019m tired of reading whataboutism on this case, so I fed the released transcripts to Claude to check if there was finally an explanation for the data anomalies. LOL. You know the answer.&#8221;<\/p>\n<p class=\"para\">Yes. We know the answer to whether there\u2019s finally an explanation for the data anomalies in podcasts that have said we are not yet getting to the explanation for the data anomalies, but stay tuned. We will get there, and then we\u2019ll see if Claude will agree with the conclusion that I\u2019ve drawn: that this prosecution was based on a mistake.<\/p>\n<p class=\"para\">Thanks for listening.<\/p>\n<p class=\"para\">I hope we can get to the next episode soon.<\/p>\n<p class=\"para\">This is Larry Lessig.<\/p>\n<\/div><footer class=\"ep-footer\"><a class=\"ep-footer-link\" href=\"#gino-index\">\u2191 Back to index<\/a><a class=\"ep-footer-link next\" href=\"#gino-episode-4\">Next: Allegation #2 \u2192<\/a><\/footer><\/section>\n<section class=\"episode\" id=\"gino-episode-4\" data-screen-label=\"04 Allegation #2\"><header class=\"ep-header\"><div class=\"ep-eyebrow\">Episode 04 &middot; The Law, Such As It Is &middot; Season 3<\/div><h2 class=\"ep-title\">Allegation #2<\/h2><p class=\"ep-blurb\">A close look at the evidence behind the second of four allegations of data fraud.<\/p><\/header><div class=\"ep-body\"><div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para has-dropcap\">This is Larry Lessig. This is the fourth episode of the third season of the podcast, &#8220;The Law Such As It Is.&#8221; In this season, we are focused on the tenure removal of Francesca Gino, the Harvard Business School professor accused of academic misconduct.<\/p>\n<p class=\"para\">The charges against Francesca Gino were grounded in four papers \u2014 out of the more than 140 that she had published in the 20 years of her academic career.<\/p>\n<p class=\"para\">These four papers presented what we\u2019re going to call &#8220;data anomalies,&#8221; meaning that the publicly posted data doesn\u2019t quite make sense as the original data that would have supported the analysis of each paper.<\/p>\n<p class=\"para\">The question is why these anomalies exist. Are they mistakes\u2014 made by the people preparing the data to be analyzed\u2014or are they intentional fabrications? And more specifically, in legal terms, was there &#8220;clear and convincing evidence&#8221; that they were intentional fabrications?<\/p>\n<p class=\"para\">Remember where this started: Data Colada \u2014 a fantastic group of data scientists, testing the validity of published empirical research \u2014 discovered what they called &#8220;apparent anomalies&#8221; in these four papers. That led them to submit a complaint to the Harvard Business School. That complaint began the two-step process that eventually took Francesca\u2019s tenure away.<\/p>\n<p class=\"para\">In the first step, the Harvard Business School investigated the four studies in the four papers in question. They concluded that the anomalies were intentional\u2014that she had committed academic fraud by intentionally manipulating the data in the way the anomalies had revealed. HBS then recommended that the university convene the second step in this process: what\u2019s called the &#8220;Third Statute&#8221; proceeding by the university to revoke her tenure.<\/p>\n<p class=\"para\">We\u2019re here, with this podcast, because the university did indeed convene the Third Statute procedure and then found Francesca guilty, making her the first tenured professor in Harvard\u2019s history to have her tenure revoked.<\/p>\n<p class=\"para\">But as we covered in the first three episodes, in the first stage of that process, the HBS internal investigation, Francesca was blocked from gathering evidence to defend herself: HBS forbade her from discussing the charges against her with anyone except the two advisors she was permitted to select. That meant she could not talk to the people who would have prepared the data that she had analyzed to figure out exactly how the anomalies could have been produced. It also meant she was forbidden from hiring a forensic expert who could help her to evaluate the forensic evidence that HBS\u2019s own forensic expert was developing to establish her guilt. HBS, in other words, got expert help preparing the prosecution; Francesca was denied expert help in mounting a defense.<\/p>\n<p class=\"para\">And as we\u2019re about to see in this episode, that denial of process was fatal to her ability to demonstrate her innocence. Because the mistakes made by the forensic expert hired by the business school led the Investigative Committee of the business school \u2014 what we\u2019ll call, the &#8220;IC&#8221; \u2014 to conclude that she was guilty. And yet, when those mistakes were identified by the forensic expert Francesca was able eventually to hire, HBS abandoned that evidence, and hired a di\ufb00erent expert to craft a whole new case against her.<\/p>\n<p class=\"para\">I make lawyers for a living. Let\u2019s just name what this is. This is a pathology of lawyers as advocates, rather than lawyers as justice seekers. The business school had gaggles of lawyers lined up to prosecute Francesca. But apparently, no one had the courage to say \u2014once the evidence that had grounded her initial finding of guilt evaporated\u2014that maybe they should reconsider whether in fact she was guilty. Instead, rather than reconsider, they doubled down.<\/p>\n<p class=\"para\">All that\u2019s a bit abstract. Let\u2019s begin in this episode with the clearest example of this fatal flaw: what we\u2019re going to refer to as allegation number two.<\/p>\n<p class=\"para\">To tell this story, I\u2019m going to have the help of a friend \u2014 actually two friends. The first friend is a friend who is expert in data analysis, and who can help unpack the facts behind the charges made against her.<\/p>\n<p class=\"para\">My conversation with this friend is going to be a little bit analysis, a little bit recounting of what actually happened. But the aim is to help you to see just how weak the charges against Francesca are. So weak that certainly no one could conclude fairly\u2014at least with &#8220;clear and convincing &#8220;evidence,&#8221; as the rules required\u2014that she is guilty of academic misconduct. And by the time we get to the end of this story, I think you will agree that she is, as I believe and as she fiercely insists, actually innocent.<\/p>\n<p class=\"para\">But for complicated reasons unrelated to this case, my friend is not free to talk freely about the case. So I can\u2019t interview them directly for this podcast directly.<\/p>\n<p class=\"para\">That\u2019s why I need a second friend, to give voice to the story that my first friend will help tell. That second friend is Ron Suskind. Ron will be the voice of my anonymous first friend.<\/p>\n<p class=\"para\">Ron is a journalist, an author, a documentary-maker, and for the past few years, the president of an AI company, Sourcebase.ai. From 1993 to 2000, he served as senior national a\ufb00airs writer for the The Wall Street Journal, and in 1995 won the Pulitzer Prize for Feature Writing for a series of articles on inner-city students. His books \u2014 including A Hope in the Unseen, The One Percent Doctrine, Confidence Men, and Life, Animated \u2014 have been wildly successful best-sellers. I\u2019ve known Ron for 14 years; he\u2019s been a dear friend, with endless advice, some of which I\u2019ve taken.<\/p>\n<p class=\"para\">Welcome, Ron, and thank you for doing this. Tell us a bit about Sourcebase.ai.<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">Well, it\u2019s really the AI for media. A couple years ago, I gathered leaders in AI and leading journalists some of whom you may know to build an AI that can essentially be a truth discovery and distribution machine scalable frankly to data pools as large as the Pacific Ocean.<\/p>\n<p class=\"para\">And what it is is an AI that lots of folks in media are using as well as citizens where can essentially have a conversation with enormous pools of document data \u2014 evidence, essentially \u2014 and discover what is true. We\u2019re having a crisis of misinformation, I would submit, in this period. And we would be considered a counter force to that by virtue of being able to use AI to tap into essential fact and evidence instantaneously.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And it\u2019s a rag-based AI.<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">It is. It is an LLM rag that means a large language model integrated with retrieval augmented generation \u2014 RAG. We\u2019re, I think, maybe the most powerfully scalable rag with all sorts of added capacities for precise research and analysis, and that\u2019s why it is one that our friends in the media are using as well as folks in government, researchers, and citizens as well.<\/p>\n<p class=\"para\">Okay, so this is not as sophisticated an e\ufb00ort to tell the truth, and we\u2019re not using AI; we\u2019re just doing it person-to-person right here.<\/p>\n<p class=\"para\">Well, let me just say that what you have here though is the kind of thing that, of course, we see as powerful in its evidentiary quality. This test on testimony here, this conversation if you will, with the testimony of your friend who I will play in this in this episode, is the kind of thing, of course, that we put up on Sourcebase.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">We\u2019re going to explore how much we can put on to Sourcebase, but right now for this episode, this is how it\u2019s going to work.<\/p>\n<p class=\"para\">I asked my first friend, the anonymous friend, to write answers to the questions I would pose in this podcast. They did. And now I\u2019m asking my second friend, Ron, to read those answers. The aim is for Ron to be the voice of my unnamed first friend. The words he will utter are not his words; they are my friend\u2019s words. Ron will give my anonymous friend a voice. It\u2019s a little bit weird, but Ron\u2019s got a pretty good voice, and for a Pulitzer Prize winner, he reads pretty well.<\/p>\n<p class=\"para\">So, thank you, Ron, for playing this role.<\/p>\n<p class=\"para\">You\u2019re supposed to say, &#8220;Thank you, professor,&#8221; at this point.<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">Thank you, professor. I suppose that\u2019s what your friend would have said to you. That\u2019s what I\u2019d say. Okay, well, let\u2019s go Larry.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so let\u2019s jump in.<\/p>\n<p class=\"para\">At issue in Allegation 2 is a paper by Francesca and two coauthors, one a professor at Kellogg and one at Columbia, &#8220;The Moral Virtue of Authenticity: How Inauthenticity Produces Feelings of Immorality and Impurity.&#8221;<\/p>\n<p class=\"para\">What was this paper about? What were the authors trying to demonstrate?<\/p>\n<p class=\"para\">The hypothesis of the paper is that experiencing inauthenticity creates feelings of impurity, which lead to a desire for physical cleansing and greater prosocial behavior as a form of moral compensation. So in simpler terms, when you do something &#8220;inauthentic,&#8221; like acting enthusiastic about a project you secretly dislike, you manifest a desire to cleanse yourself or &#8220;make up for it&#8221; by doing something good for others.<\/p>\n<p class=\"para\">This study was conducted partly in the lab (meaning in a physical space where people came into a lab, sat at computers, and answered questions that were presented on the computers) and partly online. Why did they divide the study like that?<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">It seems the lab couldn\u2019t get as many participants as the researchers wanted for sample size purposes, so they sought more participants online.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">The original charges by Data Collada focused on 20 anomalous entries that came from the online participants. Can you describe what those anomalous entries were, and why they seemed weird.<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">The study was supposed to be limited to Harvard College undergraduates. One of the demographic questions was &#8220;year in college,&#8221; which is a question that would make perfect sense for undergraduates. Good answers could be &#8220;freshman,&#8221; &#8220;sophomore,&#8221; &#8220;junior,&#8221; or &#8220;senior.&#8221; Maybe &#8220;class of 2015&#8221; would be a reasonable answer, or something like that. But Data Colada noticed that in the published data for this study, 20 of the entries had &#8220;year in college&#8221; set to &#8220;Harvard.&#8221; That made no sense: no Harvard College undergraduate would enter that value, Data Colada reasoned. Certainly, twenty di\ufb00erent Harvard College undergraduates wouldn\u2019t answer that way.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, but why did that suggest that Francesca might have been engaged in academic misconduct?<\/p>\n<p class=\"para\">Well, if real Harvard students didn\u2019t enter this data, then these answers must have come from someone else. Then you have to ask who? Maybe it was Francesca who provided these answers herself, by just faking data, by just typing it in. As Data Colada explained their thinking, and I quote: &#8220;it is di\ufb03cult to imagine many students independently making this highly idiosyncratic mistake.&#8221;<\/p>\n<p class=\"para\">Let\u2019s unpack a bit more the particular way that these entries might have been created. First, how were participants recruited? Why would they participate?<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">Some were just interested in the research. But some did it for the money. In the physical lab, there was a well-defined process for paying research participants: cash was given to each participant at the end of the study. Online, participants were given Amazon gift cards. The pitch to prospective subjects was along the lines of: &#8220;Do this easy survey, and get a $10 gift card.&#8221;<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And so, how do people learn about these types of surveys?<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">Some learn about them from professors at Harvard College announcing them in class. Some learn about them from email lists that are supposed to be limited to Harvard College students, such as mailing lists for undergraduate dormitories.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So for the online participants, someone would learn that there\u2019s an opportunity to earn an Amazon gift card by filling out a survey, and they would follow a link to fill it out. Right?<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">Right.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Now when they logged on to the platform to fill out the survey, the platform (which was served by a company called Qualtrics) would have collected a bunch of data about those people filling out the survey, right?<\/p>\n<p class=\"para\">Yes, Qualtrics gathered the IP address for the location from which the survey was entered, the &#8220;user-agent&#8221; of the user\u2019s browser (which is meant to report which browser the user is using, like Chrome versus Firefox, and what version, and also what operating system), as well as some other data.<\/p>\n<p class=\"para\">And so, then, what was the theory about what Francesca must have done to create these 20 anomalous entries in the dataset?<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">The simplest theory is that Francesca wasn\u2019t getting the results she wanted with her core dataset, so she added these 20 entries to bolster the results.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">But what is the theory? That Francesca doesn\u2019t know what &#8220;year in college&#8221; means? That she\u2019s confused about what the form is asking for? Didn\u2019t she design the survey to collect the data?<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">Right, it\u2019s her survey. She knows what the question asks, she knows what a logical answer would be. If she were stu\ufb03ng the ballot box, so to speak, you\u2019d think she\u2019d be careful enough to submit correct answers \u2014 like &#8220;sophomore,&#8221; or &#8220;junior&#8221; \u2014 definitely not &#8220;Harvard.&#8221;<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Now wouldn\u2019t you expect that any reasonable investigator would look at this and think it\u2019s kind of weird to imagine Francesca is the source of these anomalous entries\u2014again, because if she were the source of these anomalous entries, she at least would have filled out the survey form correctly.<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">Right. If I were an investigator, I\u2019d be asking questions here. Francesca wouldn\u2019t make this sort of mistake. No one who knows undergraduates, who has attended or spent time at a US college and speaks English even moderately competently, would think these answers make sense.<\/p>\n<p class=\"para\">So, if you were a careful investigator noticing the anomalies in these anomalous entries, you could have used the Qualtrics database to help you learn something about where these entries came from, right?<\/p>\n<p class=\"para\">Yes, Qualtrics has great tools for investigating here. Broadly, this is in the category of meta-data, which means data about data, like the last-modified date of a Word file on your hard drive. Qualtrics gathers data like this automatically, as the user submits each survey entry.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">What would this metadata have shown? Or more precisely, what would an investigator have imagined the metadata should have shown?<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">The smoking gun here would have been an IP address that was, for example, tied to the IP address at Francesca\u2019s home, or maybe her o\ufb03ce, or her cell phone, or a friend\u2019s house. Whatever, it would link back to a location that Francesca would have been associated with at a time when Francesca would have been there.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And so if the metadata had shown a place where Francesca was, or reasonably could have been, that would have supported the conclusion that Francesca made these 20 suspicious entries. Correct?<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">Right. If the suggestion is that she did it, then the IP addresses should point to her.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">From your understanding of the investigation done by the business school: Did they consult the log files or attempt to determine forensically where or who could have made these entries?<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">No. In fact, nowhere close. But there\u2019s a bit of a story here.<\/p>\n<p class=\"para\">By default, Qualtrics output files don\u2019t include the IP address of the user submitting each survey, nor the user-agent header of the user\u2019s browser. So an unskilled investigator might go to Qualtrics, download the file, see no IP address or user-agent information, shrug, and say it can\u2019t be done. &#8220;It can\u2019t be done&#8221; \u2013 because the data had to be captured at the time, it can\u2019t be reconstructed, you can\u2019t guess what IP address or user-agent a particular user had sometime in the past. And if it\u2019s not in the file, an unskilled investigator might think, then it wasn\u2019t captured, and it can\u2019t be obtained now.<\/p>\n<p class=\"para\">But in fact, Qualtrics always captures this data. You just need to know how to ask for it. Basically, you just have to tell Qualtrics to include this metadata in the download file. I was actually the person who figured this out, not by being super clever, but by just reading the Qualtrics\u2019 manual. It seems the HBS investigators didn\u2019t read the manual, because their report didn\u2019t have that data.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Was there something else about these 20 entries that Data Colada also flagged?<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">Right. Not only did the entries have weird entries for the student\u2019s year, they all seemed to have extreme values (like 7\u2019s) that strengthened the ultimate conclusion of the study. Not all of them had extreme values. In fact, one row had extreme values in the other direction \u20141\u2019s rather than 7\u2019s. But on balance, they did strengthen the result, though not by much. In fact, as Francesca\u2019s experts later demonstrated, removing the 20 entries didn\u2019t a\ufb00ect the statistical significance of the paper at all. But it is true that except for one row, they all did point in the same direction.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">That is weird. Is there any innocent explanation of how the extreme values might have been produced?<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">Turns out there is. People doing a survey in a hurry often choose extreme values. For example, if you do a survey with your keyboard, not mouse, because you realize that\u2019s faster, you might tap tab to go to the next question and choose the same number for every question. Or if you are using a mouse, just click click click straight down in a straight line, again yielding the same answer.<\/p>\n<p class=\"para\">So someone just hurrying through the survey could produce these extreme answers because that could be the fastest way to get through?<\/p>\n<p class=\"para\">Right.<\/p>\n<p class=\"para\">Ok, so these were the problems with these 20 anomalous entries. The business school investigation of them was incomplete \u2014 it didn\u2019t know where they came from. But in the first interview they had with Francesca, on February 28, 2022, they presented the entries to Francesca to get her to explain how they could have been produced. Did she, on the spot, have any way to explain them?<\/p>\n<p class=\"para\">No. On the spot, she didn\u2019t have an answer. She did say that the errors were not so obvious that she would certainly have caught them. The &#8220;Harvard&#8221; rows are strange when considered on their own. But she didn\u2019t think she would have noticed them spread among all the others, especially given that this column of data was not even used in the analyses.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So she couldn\u2019t explain how they were produced. Did the investigative committee (the &#8220;IC&#8221;) of the business school, forensically, determine whether she had in fact produced them\u2014by tying their submission to her machine, or her location, or anything like that?<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">No. Instead, they made this an inference without evidence: There were anomalies; she would benefit from the anomalies; therefore, she must have created the anomalies. She, in other words, must have committed academic misconduct \u2013 fraud.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">After the conclusions of HBS\u2019s IC were made public, Francesca was free to retain her own experts. One of the first things her experts did was to try to unpack what really produced the 20 anomalous entries. Can you describe a little bit about how they would have done that?<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">Well, the first step was to use the Qualtrics dashboard to produce a report with the metadata, the IP addresses and user-agent headers, and then download that report and look at those 20 weird rows with &#8220;Harvard&#8221; as the year.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And when they did that, what did they conclude?<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">Two experts looking at this concluded the same thing. Basically, that there was no way Francesca had done this.<\/p>\n<p class=\"para\">First, the entries all came from a weird, out-of-date browser\u2014 a Firefox version 5 browser running on a 2001 Windows XP computer. Almost no one was using Windows XP in 2014. Francesca was always a Mac user. Firefox 5 was released in 2011; by 2014, Firefox was up to, I am not making this up, version 27. From the start, Firefox checked for updates when it was launched, making it unusual for old versions to stay \u2013 even before Firefox added an auto-updater (in version 15). Bottom line is this user-agent line doesn\u2019t look plausible as any machine that Francesca could have been using. It didn\u2019t look like a real user in 2014; it certainly didn\u2019t look like Francesca.<\/p>\n<p class=\"para\">Second, the 20 Harvard submissions came from 20 di\ufb00erent IP addresses, mostly in other countries, mostly in data centers, meaning from servers, not the sort of computer Francesca had access to. None of the connections came from any place Francesca might have been.<\/p>\n<p class=\"para\">Third, each entry ends before the next one begins. There was never any overlap, not even by one second. Real users would overlap.<\/p>\n<p class=\"para\">Fourth, each entry specifies a di\ufb00erent email address. But none of the email addresses exist anywhere else on the web. That\u2019s very unusual: normal users leave digital footprints. These addresses were all from free email services, like Gmail or outlook.com.<\/p>\n<p class=\"para\">Fifth, there were some patterns in the submissions, suggesting the user was using the browser\u2019s autofill to copy answers from prior form submissions. The same capitalization repeated in some, then changed to something else in multiple subsequent entries.<\/p>\n<p class=\"para\">Putting all those together, the picture was pretty clear to me. This was one attacker, using a form-filler and &#8220;proxies&#8221; to bounce tra\ufb03c through servers in other countries, and using a &#8220;proxy switcher&#8221; function to quickly change from one proxy to another after each submission. The person was not a genius, because they didn\u2019t even try to deeply cover their tracks. But they covered enough to trick the system into saying the submissions were OK.<\/p>\n<p class=\"para\">There was therefore no direct evidence \u2014 as opposed to an inference from motive \u2014that this person was Francesca. By far, the better theory was that it was just a scammer trying to get some free Amazon gift cards.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So once Francesca\u2019s experts got to look at the data, they were able to show absolutely that the charge that she had fabricated these entries was just baseless. And indeed, eventually, the hearing committee of the university, \u2014 the committee that would determine whether to remove her tenure \u2014 agreed. They saw her expert evidence, and they agreed that the 20 anomalous entries were not evidence of fraud or any other misconduct by Francesca. They agreed that this was a product of\u2014as they called it\u2014a &#8220;scammer.&#8221;<\/p>\n<p class=\"para\">For me, this was just one more &#8220;holy shit moment&#8221; among many in this case.<\/p>\n<p class=\"para\">The whole of the charges made against Francesca by Data Colada with respect to this paper were tied to these 20 anomalous entries.<\/p>\n<p class=\"para\">Data Colada had noticed them. It had flagged them. It had asked the business school to investigate them. All that was completely fair.<\/p>\n<p class=\"para\">But then the business school botched that investigation. Even more astonishingly, it blocked Francesca from conducting her own investigation before it concluded that she was guilty. Based on that conclusion of guilt, it asked the university to revoke her tenure. And yet once Francesca had expert support, the kind of forensic support that the business school itself was using to prosecute her, she was able to establish absolutely that she was not responsible for these anomalous entries. Is that a fair summary?<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">Yes, the 20 supposedly suspect &#8220;Harvard&#8221; entries were fully resolved.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">I remember learning all this. Like you, I had not known anything about the prosecution against her before the business school announced publicly that she was guilty. But afterward, I recall learning that the whole basis of her guilt with respect to this charge at least had been blown up. That these anomalous entries had nothing to do with her.<\/p>\n<p class=\"para\">And I remember thinking at that point, &#8220;Well, this is all going to be over. The business school will realize they prosecuted her on the basis of a mistake. Maybe they would reflect on their own procedural flaws that helped produce that mistake. Maybe some contrition would have manifested. But certainly, at least with respect to this paper, this process would be over.&#8221;<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">Almost. There was a second finding that the IC made about this paper.<\/p>\n<p class=\"para\">Based on a report by HBS\u2019s forensic expert, Maidstone, the IC said, I quote, some data in the OSF dataset do not appear in either of the two Qualtrics datasets for this study, that those data strongly support the hypothesized and reported results, and that some data in the two Qualtrics datasets do not appear in the OSF dataset.<\/p>\n<p class=\"para\">That\u2019s the end of the quote. OSF stands for Open Science Framework, which is an online platform where researchers can openly share data, methods, and results to make science more transparent. Francesca had deposited the data for this study on a public platform. The IC thought they had identified a gap between the Qualtrics data and public data that could not be explained.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So, the suggestion was that Francesca must have edited the data to delete observations that weren\u2019t helpful to her conclusions, right?<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">Right. That was the theory, and that\u2019s what the IC concluded.<\/p>\n<p class=\"para\">But here again, it turns out, HBS\u2019s original expert\u2019s report was just wrong.<\/p>\n<p class=\"para\">After the IC concluded that Francesca was guilty, based on this expert report, the experts changed their minds. In December 2023, 9 months after the business school had declared to the world that Francesca was guilty, the experts issued a new report. That report said, in e\ufb00ect, &#8220;oops, we were wrong. There were no missing entries. Every complete response in Qualtrics had a match in the OSF dataset. And every respondent in the OSF data has a match in Qualtrics.&#8221;<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And so did the IC then update their report, based on the correction that their expert had made?<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">No. The IC made no corrections to their conclusions. They had relied on the claim that the data was missing. When Maidstone revealed there was no data missing, they said nothing.<\/p>\n<p class=\"para\">Worse than that, HBS ended up with two parallel processes. Remember the IC finding was from March 2023. In June 2023, HBS filed a complaint with the university, starting the process to remove Francesca\u2019s tenure. In December 2023, Maidstone told HBS that rows weren\u2019t missing after all. But by that point, the tenure process was already underway. HBS didn\u2019t alert the Hearing Committee that the initial Maidstone report was wrong, and Maidstone had made a correction. The prosecution continued without once acknowledging it was based on a mistaken finding by HBS\u2019s experts.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Let\u2019s just make the point once again: Here was a second mistake in the business school\u2019s &#8220;expert&#8221; report. The first was its failure to notice that the 20 entries were by a scammer, not by Francesca; the second was its claim that there were entries that had been omitted from the Qualtrics dataset; once it checked its work, the experts discovered this too was wrong. There were no missing entries. Both times, the IC made an error that Francesca could have corrected, if she had had her own expert support before HBS concluded she was guilty.<\/p>\n<p class=\"para\">But she didn\u2019t, because the rules that the business school imposed upon her forbid her from having expert support before it concluded she was guilty. It\u2019s like there\u2019s actually a reason why you allow the defendant to provide her own evidence \u2014 and test the evidence of the prosecution\u2014before you ask the jury to decide whether she is guilty. It\u2019s Due Process 101.<\/p>\n<p class=\"para\">In the tenure proceeding, HBS obviously realized that the scammer story and the missing rows that were no longer missing were fatal to the charges standing behind Allegation 2. They were the entire Allegation 2 charge. HBS could have dropped the charges, or gone back to the IC and asked them to reevaluate the evidence. But instead, as I\u2019ve described, they doubled down: they sent the paper out to a new expert to see what new problems the new expert might be able to identify.<\/p>\n<p class=\"para\">That new expert then examined the data further and called out a di\ufb00erent issue altogether. This charge involved 21 rows of data, some of which were the Harvard entries, and some were the rows Maidstone thought were missing. Specifically, within those rows of data, he found 154 cells that seemed to have been changed between the source data and the data used to conduct the analysis. Based on his analysis, the hearing committee\u2014the committee that would decide whether to revoke Francesca\u2019s tenure \u2014 concluded that, quote: &#8220;all of the alterations are in the direction of the paper\u2019s hypothesis.&#8221;<\/p>\n<p class=\"para\">First, what does that mean?<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">If it were true, it would mean that if you examine the changes between the two data files, they all would have made the claims of the paper stronger. Statistically stronger. Again suggesting a motive for Francesca making the changes.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Well, is that true? Is it true that &#8220;all of the alterations are in the direction of the paper\u2019s hypothesis&#8221;?<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">No, it is not true. Not all of the 154 changes strengthen the claims of the paper. At most, half of them do. But half of the changes are in columns that weren\u2019t used in the final statistical analysis. So on the theory that Francesca was tinkering with the values to strengthen the conclusions of the paper, you\u2019d have to explain why half of that tinkering was with data that didn\u2019t matter to the conclusions of the paper.<\/p>\n<p class=\"para\">Francesca\u2019s lawyers made this point at the tenure revocation hearing repeatedly. The Hearing Committee apparently just missed it.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">This is critically important. The whole of the Hearing Committee\u2019s argument resolves to this: &#8220;there are changes in the data, all of those changes favor the conclusion of the paper; only Francesca had an interest in making changes to strengthen the conclusion of the paper; so therefore, Francesca must have made the changes.&#8221; The whole of their argument, in other words, is an inference from motive.<\/p>\n<p class=\"para\">But in racing to their conclusion, the committee missed a pretty important fact about motive: not all of the changes support the conclusion of the paper. Which raises a pretty obvious question about motive. On the theory that Francesca made these changes intentionally, why would half the changes she made have nothing to do with strengthening the conclusions of the paper?<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">Why exactly.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, I want to flag this flaw, because we\u2019re going to see it with each of the four allegations made against Francesca. In each of the four allegations, we\u2019re going to see that the hearing committee concluded that all the anomalies that they were talking about, the changes that they had identified, strengthen the conclusions of the papers Francesca ultimately published. In each case, that claim is simply not true.<\/p>\n<p class=\"para\">In every single case, some of the anomalies strengthened the finding, while some had no e\ufb00ect at all.<\/p>\n<p class=\"para\">So in all four papers, the real question about motive should be this: If Francesca really was intentionally manipulating the data to strengthen her conclusions, why would she make changes to the data that both strengthened the results and didn\u2019t strengthen the results? Why would she waste her time like that?<\/p>\n<p class=\"para\">But let\u2019s return now to the specific changes that were made with this allegation \u2014 or that remained, after the scammer charges are gone, and after the missing entries charges are gone.<\/p>\n<p class=\"para\">As I said, the hearing committee said there were 154 changes. There were actually 160, but we\u2019ll stay with the HC\u2019s number. Half of them strengthened the results\u2014marginally\u2014and half did not. But for the half that did, that sounds really bad, right.<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">It\u2019s not great. Data values should stay the same from beginning to end. In this case, there was a clear di\ufb00erence between the original submissions and the final data set, and, on the surface, no obvious way to explain it.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Well, the business school thought it had a way to explain it, right?<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">Yes they did. Their hypothesis again was that Francesca had made the changes intentionally, by working through the data file, and changing 154 cells to strengthen the hypothesis of the paper.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">154 cells out of how many? I mean, how many cells could have been changed to strengthen the hypothesis of the paper?<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">Well, there were 10,311 cells of data in the spreadsheet. Only some of those cells were relevant to the paper\u2019s analyses. 2,455 cells had results that could have been changed to strengthen the hypothesis of the paper.<\/p>\n<p class=\"para\">So that means, if she was seeking to strengthen the hypothesis of the paper, there were 2,455 possible cells that could have been changed, right?<\/p>\n<p class=\"para\">Right<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">But of these 154 changes, how many were changes that actually strengthened the conclusions of the paper?<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">80 of 154 strengthened the conclusions of the paper.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">80 of the 154 strengthened the study; 74 did not. Ok, and so as you studied the 154 changes, did it seem like the cells that were changed were just randomly chosen? Did it seem like someone was just working through the spreadsheet, tweaking the results to strengthen the conclusions of the paper?<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">No it did not. Because as I looked at the changes, I noticed a pattern. These weren\u2019t just randomly selected cells that were di\ufb00erent. It didn\u2019t look like someone was just moving through the spreadsheet, tampering with the values, asking &#8220;what data would be good in this row to strengthen my hypothesis.&#8221; Instead, the changes had a pattern. It looked like the data had gotten shu\ufb04ed, or swapped from one part of the spreadsheet to another.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So how could that swapping have happened, if not intentionally? I mean, Excel data doesn\u2019t just get up and walk around a spreadsheet, does it?<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">At first I had no idea. The values in a row should stay in that row. Row 1 data can\u2019t just appear in row 9, and vice versa. But when I looked deeper into the mechanics of Excel, I was kind of surprised to learn about a feature that Excel calls &#8220;Cut and Insert.&#8221; That\u2019s not a terribly helpful name. If I were naming it, I\u2019d call it &#8220;Swap.&#8221; But basically, the way it works is this: if you highlight a range of cells, press the shift key, and then drag the highlighted range, Excel will take the selection you started with, and put it where you drop it. That part seems intuitive, like a drag or move. Much less intuitive is this: Excel will also take the block of cells where you dropped it, and move them to the place you started. It essentially swaps the source range with the destination range. The two sets of values trade places.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Wait. So if you highlight a 2&#215;2 block of cells \u2014 A1 to B2 \u2014 and then shift-drag that block to C8, the values in A1 to B2 will now be in C8 to D9. But what you\u2019re saying is that whatever was in C8 to D9 originally, is now in A1 to B2?<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">Exactly. The two blocks are swapped. Regardless of how many cells you\u2019ve highlighted to move, a block of the same size gets swapped with the block you\u2019re dragging.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Why would Excel even have a feature like this?<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">Well, Excel is kind of a Swiss army knife of data tools. What features doesn\u2019t it have? Actually, I have wanted this feature at a couple points in my time working with Excel, but I didn\u2019t know what it was called or what its shortcut key was. And since I learned about it, I have actually used it a few times. So I think it is an appropriate feature to include in Excel.<\/p>\n<p class=\"para\">But it\u2019s a dangerous feature. Excel doesn\u2019t pop open a dialog box to say &#8220;You are swapping 3 cells with 3 other cells,&#8221; or 300 cells with 300 other cells, as the case may be. Excel just does it. Excel\u2019s online help does warn that &#8220;Excel doesn\u2019t warn you.&#8221; But if you haven\u2019t been warned, you\u2019re not likely to notice the swap. Instead, if you\u2019re selecting a range of 8 rows by 9 columns to look at as you check data for problems such as inattentive users, you might not notice that a block of 8 rows by 9 columns has been swapped with another block. Especially when all the data kind of looks the same, all survey data, in this case all single-digit numbers from 1 to 7.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Ok, tie all this together for me. Why does this swapping function matter to the story about these 154 cells?<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">Well, this feature actually could explain a lot about the 154 anomalous data entries.<\/p>\n<p class=\"para\">One area of anomalies is a block of cells 9 columns wide by 8 rows deep. When I looked at the data, I realized this 9&#215;8 range had been swapped with a di\ufb00erent 9&#215;8 range. And when I realized that, this swapping function became a suspect. When I realized that this range had been swapped, it became plausible that this hidden Excel feature was actually responsible for the change. If the rows and columns had been arranged in a particular way \u2014 and actually, very similar to the way they were arranged in one version of the files found on Francesca\u2019s hard drive\u2013then the 9 columns and 8 rows that I had found were swapped could have been unintentionally swapped by this hidden Excel feature.<\/p>\n<p class=\"para\">And that single unintentional swap would then account for 94% of the changes observed in the file. Of these changes, 44% were changes irrelevant to the study hypotheses. And another 11% were changes to a measure that was the average of other measures a\ufb00ected by changes, so it would have automatically changed as a result.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So one inadvertent (or unexpected) swap, caused by shiftdragging a block of cells rather than just selecting a block of cells, could explain 94% of the anomalous entries?<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">Yes, it could.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">What about the other 6%?<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">They can\u2019t be explained by the hidden swap feature. But they could have been produced by another hidden Excel feature. This command is Copy and Replace, Control-Drag. This copies cells, meaning creates a second instance of the same data, but in a new place, overwriting what\u2019s there, again without a warning message. There are two other blocks of cells, for which Copy and Replace would fully explain the observed changes. And then, yes, every change from the original file to the posted file \u2013 would all be totally explained.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So one hidden feature of Excel \u2014 the swap feature \u2014 could explain 94% of the changes. The other hidden feature \u2014 this copy and replace, control-drag feature done two times, could explain the rest. And both of these unexpected changes would happen simply because a shift key, or a control key, was inadvertently depressed as the person working with the file was moving data to line up the columns to be analyzed.<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">Right. The three inadvertent edits together would explain each of the 154 changes, and thus create a plausible explanation for how the changes actually happened. Meaning, whoever was &#8220;cleaning&#8221; the data file could have been a bit sloppy with how he or she was using the keyboard and mouse, inadvertently producing exactly these changes. Not by randomly tweaking the data to strengthen the results \u2014 these changes were not randomly distributed \u2014 but by these hidden Excel features that are, as you inadvertently trigger them, easy to overlook.<\/p>\n<p class=\"para\">Given this pattern in the changes, this alternative explanation seemed wildly more plausible to me than the idea that Francesca was simply tinkering with the data to produce a stronger paper.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">But why would anyone be making any changes to the data at all? Why don\u2019t you just move from the surveys \u2014 in lab and online \u2014 to the data that gets analyzed?<\/p>\n<p class=\"para\">In this case, there were both lab participants and online participants. The data from the two groups arrived in two di\ufb00erent files. Those files had to be merged. That merging sounds easy, but the columns don\u2019t line up because the two surveys had di\ufb00erent fields due to di\ufb00erent setups. So an RA would have had to copy the rows into a single file, rearrange columns, and get it all lined up.<\/p>\n<p class=\"para\">Then there\u2019s the process of data cleaning. This is empirical work with data from survey respondents. Survey respondents are messy, to put it politely. RAs look for incomplete answers, people who did not take the study seriously (by writing bogus essays, for instance) or people who were obviously confused or didn\u2019t even finish the task. Those would be flagged and possibly excluded. It\u2019s a lot of data massaging. We could imagine a super-RA doing it all perfectly. But it\u2019s also easy to imagine an RA making a mistake.<\/p>\n<p class=\"para\">And again, you said RA. You don\u2019t believe Francesca was doing this massaging, right?<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">I don\u2019t know, I wasn\u2019t there. But a HBS professor getting columns in a spreadsheet lined up? And merging files? It seems way above her pay grade. More like the work RAs do.<\/p>\n<p class=\"para\">And indeed, that\u2019s precisely what Francesca told HBS: As she told them, the data merging and cleaning was not done by her, it was done by RAs. It was the analysis that was, and should be, done by professors. That\u2019s the norm in her field. That\u2019s what she said happened here.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Ok, so what you\u2019re saying comes down to this: Look at the pattern of the changes, and recognize first that there is a pattern. These aren\u2019t the sort of changes that would be produced by simply moving through the file, and tweaking random cells to strengthen the result. Something more general happened to produce the anomalies, and you identified what those more general steps could have been. In preparing the data, in a process that Francesca had testified included the ordinary steps to prepare the data, these hidden Excel functions could have produced these anomalous results precisely.<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">That\u2019s right.<\/p>\n<p class=\"para\">That\u2019s your read. There is one more way to see the weakness in the HC\u2019s conclusion, at least it seems so to me.<\/p>\n<p class=\"para\">The hearing committee considered two fundamentally di\ufb00erent theories about what happened here. In one, Francesca moved through the data, intentionally making changes in 154 cells, with 80 of those changes actually strengthening the result. In the other, those changes were inadvertently produced in the process of cleaning the data, by triggering these hidden Excel features.<\/p>\n<p class=\"para\">Yes, these are the two possible theories.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">The hearing committee looked at these two theories. They concluded that there was &#8220;clear and convincing evidence&#8221; that the first theory, that Francesca had tinkered with the data to produce these 154 changes, was true.<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">That\u2019s what they concluded. They weren\u2019t super explicit in discussing the probabilities of each theory, but yes, they said the explanation was intentional manipulation, and so they didn\u2019t credit the hidden Excel features theory.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">But then here\u2019s where\u2014when I was studying this carefully in the process of writing the final appeal\u2014it struck me there was another way to see why it\u2019s option two that must be true rather than option one. And here\u2019s the argument that I made.<\/p>\n<p class=\"para\">Francesca and her experts had demonstrated a procedure \u2014 a series of steps which included these two hidden Excel features unintentionally triggered \u2014 that would have produced the 154 anomalies. Those steps are replicable; the error is replicable. Right now, we could take the original file, perform the steps constituting this procedure, and the resulting data file would have the same 154 anomalies, some strengthening the results, some not. So it\u2019s not random or magical, it\u2019s a deterministic process that gives us the anomalies that were being alleged to have constituted fraud. But on this account, they were not fraud. They were inadvertent mistakes by whoever was preparing the data file for analysis.<\/p>\n<p class=\"para\">On the HC\u2019s account, the changes were not inadvertent. They were intentional. On the HC\u2019s account, Francesca had worked her way through the data file, making 154 changes to the data. 80 of those changes strengthened the conclusions of the paper. Those 80 were among 2,455 possible cells that could have been changed to strengthen the conclusion of the paper. But regardless of the proportion that strengthened the paper, the claim is that she was simply moving through the file, tweaking the data by changing the results in 154 cells.<\/p>\n<p class=\"para\">But the weird thing if the HC was correct is this: What are the chances that Francesca\u2019s random changes of 154 cells \u2014 out of 10,311 cells \u2014 would be identical to the changes produced through the plausible procedure that included these hidden Excel features?<\/p>\n<p class=\"para\">I mean, think about it. There were 154 changes spread throughout the spreadsheet. The theory that she made those changes intentionally suggests an extraordinary coincidence: that in randomly selecting 154 changes \u2014 80 of which strengthened the result (among the 2,455 that could have strengthened the results) \u2014 Francesca just happened to make precisely the same changes that would have been produced by the plausible sequence of steps, including the hidden Excel features, that she and her experts had identified.<\/p>\n<p class=\"para\">How likely is that? How likely is it that Francesca would have randomly selected the very same cells in the very same pattern to produce the very same anomalies? What are the chances that she would have picked 154 cells exactly?<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">That is a vanishingly small probability. If Francesca were randomly modifying the file, she wouldn\u2019t happen to modify it in groups or bunches like this.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Now of course, to confirm that this alternative account \u2014 that this pattern of changes was made inadvertently, by RAs working with the data \u2014 was actually true, or even plausible, you would have to speak to the RAs. To confirm this alternative theory, in other words, someone would have to ask the RAs who actually did the work: &#8220;Could this have happened? Did you notice something like this ever happening?&#8221;<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">For sure. If I were running the investigation, I would want to talk to the RAs to see what they say about how they used Excel, and check whether that lines up with any of the patterns that seem to explain what happened to the data.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And this then brings us to the second consistent theme that we will see throughout each of these four alleged cases of academic misconduct: Did they interview the RAs who were charged with cleaning the data, and producing the files that would be analyzed by the academics writing the studies?<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">No, they did not. The business school didn\u2019t interview the people who actually manipulated the data to produce the files that Francesca analyzed for this study.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Why? Why was there no investigation involving the RAs?<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">I\u2019d say the IC thought they felt they didn\u2019t need to ask the RAs. It\u2019s the inference from motive all over again: No RA would have an incentive to manipulate data, and so if you believe the changes were intentional, there was no reason to believe they were made by the RAs.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">But that\u2019s assuming the conclusion. Because, obviously, these anomalies could have been produced intentionally, or they could have been produced inadvertently, by mistake.<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">Right.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Why didn\u2019t Francesca interview the RAs herself ?<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">Well, remember, when all this started, she was forbidden from talking to anyone except her two advisors. That gag order was lifted only after HBS had called her a fraud. Then she was gagged again during the tenure process. Her tenure lawyers did reach out to RAs as part of that process. But at that point, given the media frenzy, no one was eager to be pulled into this mess. Most RAs did not even reply to the request for an interview. One RA had her lawyer respond to convey she was unwilling to engage.<\/p>\n<p class=\"para\">That\u2019s understandable, given the firestorm. But it just emphasizes again the basic flaw in this whole process: If you\u2019re going to charge someone with academic fraud \u2014 and thereby, e\ufb00ectively end their career \u2014 you got to do the work before you announce their supposed guilt. It would have taken some work to reach out to the RAs and talk to them, no doubt. But that work is only feasible before guilt was declared. Especially in a case like this, where everything is an inference from motive. Maybe it wouldn\u2019t be necessary if the evidence was strong. Maybe if you had smoking gun evidence of manipulation\u2014like if the IP addresses for the scammer entries were actually linked to Francesca\u2019s home o\ufb03ce, or the Amazon gift cards were used to buy her a new computer. But if your case is based on inference alone \u2014 she had a motive, so she must have done it\u2014it is not unfair to insist that you do some work to provide some direct evidence to back up that inference. Especially if the standard of proof is &#8220;clear and convincing evidence.&#8221;<\/p>\n<div class=\"speaker-tag sp-ron\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ron<\/div>\n<p class=\"para\">That certainly seems right to me.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so let\u2019s pull this all together. For Allegation #2, here is what we\u2019ve got.<\/p>\n<p class=\"para\">Data Colada noticed 20 rows in the dataset supporting the claim that the paper behind Allegation 2 had anomalies within it. They\u2019re in the business of flagging anomalies, and so that\u2019s what they did. We should be happy there are people like this in the world. People like this flagging anomalies in academic research helps keep research honest.<\/p>\n<p class=\"para\">It was then HBS\u2019s job to investigate whether those anomalies were actually evidence of fraud.<\/p>\n<p class=\"para\">But as we\u2019ve seen, that process was fundamentally unfair \u2014 because it e\ufb00ectively locked the defendant in a room and denied her any e\ufb00ective support to build a case to defend herself. She was not allowed to question any of the witnesses, any of the people who actually worked on the data files at least before her guilt was declared; she was not allowed to hire a forensic expert to examine and rebut the forensic expert that the HBS was relying on to conclude that she was guilty.<\/p>\n<p class=\"para\">The investigative committee of the business school concluded she was guilty. With respect to this paper in particular, the business school concluded that she had manipulated the data, by creating these 20 rows of anomalies, to strengthen her paper\u2014and while she was at it, to earn some extra income from Amazon gift cards.<\/p>\n<p class=\"para\">But then that whole theory collapsed. That wasn\u2019t Data Colada\u2019s fault. Data Colada had no way to know the theory would collapse. They didn\u2019t have access to the information that would have shown there was no basis for believing she had manipulated this data. They didn\u2019t have access to the metadata that would demonstrate these were just scammer entries.<\/p>\n<p class=\"para\">But the business school did. Yet the completely inept investigation by the business school did not discover this before it concluded that she was guilty. Specifically, it did not figure out how to get the key metadata from Qualtrics to show she did not create these weird entries. Instead, the business school just concluded on the basis of this flawed evidence that she was a fraud.<\/p>\n<p class=\"para\">And then, once the flaw was revealed, once the mistake in their evidence was shown to them, rather than retreating, they doubled down. They sent the paper out to a new expert to search for more evidence of misconduct. The new expert discovered 154 entries in the dataset that were anomalies. On their theory of the case, Francesca had gone through the dataset and manipulated the data to produce a stronger result. They had evidence of this to support that theory, because they had two files that were di\ufb00erent. Different because of these 154 di\ufb00erent entries.<\/p>\n<p class=\"para\">The hearing committee that revoked her tenure relied on these 154 anomalies to find her guilty. It first stated that the anomalies were all in the direction supporting the paper\u2019s hypotheses. That was false. They were not. Instead, the anomalies were a weird mix: some supported the hypothesis, some did not. To assume that Francesca had produced these anomalies intentionally, you\u2019d have to explain why she manipulated parts of the data that just didn\u2019t matter.<\/p>\n<p class=\"para\">And as to the data that did matter, you\u2019d have to explain the patterns in the manipulation. On the HC\u2019s theory, she modified the data to strengthen her results, by altering 80 cells among 2,455 that could have been altered. But Francesca presented evidence of how those very same anomalies could have been produced inadvertently by the RAs preparing the file. She showed how the RAs could have followed a procedure that could have triggered these hidden Excel features, and produce exactly the pattern of changes that were identified. And so, as I argued in the final appeal, given this completely plausible way that these 154 anomalies precisely could have been produced inadvertently \u2014 in a process that was completely replicable or deterministic \u2014 what is the chance that the very same anomalies \u2014 in the same pattern and the same order \u2014 were in fact produced by Francesca randomly going through the file and tweaking 154 cells?<\/p>\n<p class=\"para\">What are the chances? Exactly zero.<\/p>\n<p class=\"para\">And yet, the HC concluded that there was clear and convincing evidence that Francesca had made these 154 manipulations intentionally. Without investigating the alternative, without interviewing the RAs who actually prepared the data, without acknowledging that in fact half of the anomalies don\u2019t even matter to the result, the committee concluded not that it\u2019s more likely than not that Francesca had committed fraud, but that her fraud had been shown by &#8220;clear and convincing evidence.&#8221;<\/p>\n<p class=\"para\">There\u2019s a technical legal term to describe all this: bullshit.<\/p>\n<p class=\"para\">That\u2019s the end of our analysis of Allegation #2. Thanks to Ron, for giving voice to my anonymous friend. Thanks to my anonymous friend, who has me the chance to voice the claim that my university has committed a fundamental injustice against one of its most promising scholars.<\/p>\n<p class=\"para\">And to those of you doing the work to understand this injustice \u2014 you, the listener \u2014 thanks for listening. Stay tuned for the next episode, when we\u2019ll turn to another of the papers she was charged with fabricating \u2014 with even less evidence than the one we\u2019ve just reviewed.<\/p>\n<p class=\"para\">One final note about Data Colada: As I said in the first episode, I have invited Data Colada to participate in these four episodes about the particular charges. I emailed each of the primary members of that collective. I have received no response from any of them.<\/p>\n<p class=\"para\">This has been the fourth episode of this season\u2019s The Law Such As It Is. These podcasts are produced by me, Larry Lessig, and produced technically by Josh Elstro of Elstro Productions. You can follow this case and find the documents related to this case on our website. That website is theginocase.info. At that website, you can also find a way to sign up for our Substack, which will echo the same material on that di\ufb00erent platform.<\/p>\n<p class=\"para\">This is Larry Lessig. Thanks for listening.<\/p>\n<\/div><footer class=\"ep-footer\"><a class=\"ep-footer-link\" href=\"#gino-index\">\u2191 Back to index<\/a><a class=\"ep-footer-link next\" href=\"#gino-episode-5\">Next: A Note on Allegation #2 \u2192<\/a><\/footer><\/section>\n<section class=\"episode\" id=\"gino-episode-5\" data-screen-label=\"05 A Note on Allegation #2\"><header class=\"ep-header\"><div class=\"ep-eyebrow\">Episode 05 &middot; The Law, Such As It Is &middot; Season 3<\/div><h2 class=\"ep-title\">A Note on Allegation #2<\/h2><p class=\"ep-blurb\">A short supplement responding to a question about the Thanksgiving 2014 timeline.<\/p><\/header><div class=\"ep-body\"><div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para has-dropcap\">This is Larry Lessig. This is the fifth episode, technically, of the third season of the podcast, &#8220;The law such as it is.&#8221; It\u2019s just a supplement to the episode we\u2019ve just released about quote, allegation number two,<\/p>\n<p class=\"para\">As we release these podcasts, I\u2019ve received lots of emails and communication from members of the Harvard faculty and the public as well. I\u2019ve said that we\u2019re going to not talk about the evidence in the case prior to completing all of these podcasts, which is taking a long time, just because this isn\u2019t my only job.<\/p>\n<p class=\"para\">But I received an email from one of the most respected members of the Harvard faculty, a friend and somebody I\u2019ve worked with in various forums for democracy reform. I was keen to dig into what he said he found when he looked at the hearing committee\u2019s report, which led him to wonder whether, in fact, we had demonstrated that hearing that allegation number two was not actually fairly found against Francesca.<\/p>\n<p class=\"para\">He pointed to one particular fact that the hearing committee had relied on that did make it seem completely implausible that anybody other than Francesca would have been responsible for the anomalies which form the basis of allegation number two. That fact is the finding by the hearing committee that the modifications of these data happened between Thanksgiving Day 2014 \u2014 that\u2019s November 27, 2014 \u2014 and the next day. The allegation is she started with a clean set of data on Thanksgiving, presumably after dinner that night, and sat down and started modifying those that spreadsheet. And by the next day, she had a spreadsheet that supported the allegations or the hypothesis of the paper more strongly. That\u2019s the basis for them believing that she had done this, because who else would have had access to the data on Thanksgiving? This is not when RAs are working. It\u2019s a very short period of time. Who else than Francesca?<\/p>\n<p class=\"para\">It\u2019s a strong argument. If it were true, one could well believe that it\u2019s very highly probable that she must have made those changes.<\/p>\n<p class=\"para\">Here\u2019s the problem with the argument: The evidence shows it\u2019s completely false. False. Because, as the expert report of Michael McGowan demonstrated, when the Business School\u2019s investigative committee went through this evidence and concluded that she modified the data between Thanksgiving and the next day, they had missed five other files in the archive that had been collected that traced the modifications of these data from September 27 \u2014  literally, two months before Thanksgiving, through Thanksgiving. There was a file from September 27 a file from September 30, a file from October 6, a file from October 7, and then the file on October, November, 26 which contains the results from the participants who took the survey, which was created then on the 27th.<\/p>\n<p class=\"para\">These earlier files trace the evolution of these data. But it\u2019s not as if the hearing committee or the investigative committee considered these five files and said these files are irrelevant for the following reasons, or they\u2019re fabricated for the following reasons, or they\u2019re made up for the following reasons. They didn\u2019t consider these data at all. What they did was ignore them. Inconvenient facts they ignored, so that they could rea\ufb03rm the false assertion that the modifications began on Thanksgiving and ended on the day after Thanksgiving.<\/p>\n<p class=\"para\">This is a common pattern in the hearing committee\u2019s report. It\u2019s kind of obvious if you read the hearing committee\u2019s report, because they don\u2019t cite a single source for their claims: They speak as if, standing on Mount Olympus declaring these truths to be true, but without pointing where in the record, the evidence is to support what they\u2019ve said.<\/p>\n<p class=\"para\">Here in particular, the lawyers had pointed them to the fact that the claim from the Investigative Committee grounding this charge on the changes occurring within the 36 hours, or 24 hours between Thanksgiving and the following day was false. Just false. But they ignored that fact. Or they overlooked it, or they were so busy they couldn\u2019t consider it, whatever the reason is, this critical fact, which my colleague pointed to, to suggest that this shows that she must have made the changes, is not true.<\/p>\n<p class=\"para\">That\u2019s all for this episode. The next episode will then pick up on one of the other three remaining charges. As you\u2019ll see as we move through these other three remaining charges, each of them has flaws as fundamental as the ones we\u2019ve identified with allegation number two. Putting them all together, with the extraordinary procedural flaws that this case evinced, should lead any fair observer to conclude that this was a mess: An outrageous mess that Harvard pushed to avoid the embarrassment of admitting that they were wrong. Because God forbid, Harvard University would be embarrassed, even if that refusal to acknowledge that they are wrong has cost an extraordinarily talented young academic her career.<\/p>\n<p class=\"para\">Stay tuned. I can\u2019t promise when, but stay tuned for the next episode.<\/p>\n<p class=\"para\">This podcast is produced by me, not my nonprofit, and working with Josh Elstro of Elstro Productions. You can find all of the episodes for this podcast on Apple podcast or Spotify, though, Spotify is a little bit clumsy in giving you access to the podcast in the right order. And you can find them on the website that I\u2019ve built for this case, TheGenoCase.info. I\u2019ve also put it on a sub stack, which you can find connected to me. Lawrence Lessig, thanks again for listening. Thanks for keeping an open mind, and thanks for the feedback, if, even if I don\u2019t have the cycles right now to respond to all of it.<\/p>\n<p class=\"para\">This is Larry Lessig. Thanks for listening.<\/p>\n<\/div><footer class=\"ep-footer\"><a class=\"ep-footer-link\" href=\"#gino-index\">\u2191 Back to index<\/a><a class=\"ep-footer-link next\" href=\"#gino-episode-6\">Next: Allegation #4 \u2192<\/a><\/footer><\/section>\n<section class=\"episode\" id=\"gino-episode-6\" data-screen-label=\"06 Allegation #4\"><header class=\"ep-header\"><div class=\"ep-eyebrow\">Episode 06 &middot; The Law, Such As It Is &middot; Season 3<\/div><h2 class=\"ep-title\">Allegation #4<\/h2><p class=\"ep-blurb\">Examining the evidence behind allegation number four.<\/p><\/header><div class=\"ep-body\"><div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para has-dropcap\">This is Larry Lessig. Welcome back to the podcast, &#8220;The Law Such As It Is.&#8221; This is episode six of season three. We are continuing the review of the charges against former Harvard Business School Professor Francesca Gino.<\/p>\n<p class=\"para\">The first three episodes of this season laid out the procedural history of a case that began with allegations by Data Colada against Francesca and ended with her having her tenure removed by Harvard University, making her (and oh, what a surprise, it\u2019s a she) the first person in the history of Harvard ever to have her tenure revoked.<\/p>\n<p class=\"para\">There were four allegations of academic fraud brought against Francesca. The fourth episode in this season laid out the evidence behind allegation number two.<\/p>\n<p class=\"para\">In this episode, we\u2019re going to consider allegation number four. As with allegation number two, and \u2014 spoiler alert \u2014 allegation number three, and allegation number one, this allegation too is astonishing in its weakness. It too does not establish, certainly with clear and convincing evidence, that Francesca committed academic fraud.<\/p>\n<p class=\"para\">Now, before we jump in, I do want to pause on the ambiguity that this way of framing the question might raise for some at least. I said it doesn\u2019t establish with &#8220;clear and convincing evidence&#8221; that Francesca committed academic fraud. Why the qualification? Why &#8220;with clear and convincing evidence&#8221;?<\/p>\n<p class=\"para\">Well, I\u2019ve said throughout this season that based on my review of the evidence adduced against Francesca and the evidence that she and her experts provided in her defense and my knowing her for more than a decade, I am absolutely convinced that she is absolutely innocent, not that she\u2019s partly innocent, not that she made some lesser o\ufb00ense, but that she committed no o\ufb00ense at all. So, it\u2019s my belief in simple layman terms, she is innocent.<\/p>\n<p class=\"para\">But whether she was actually innocent or not was not the question the Hearing Committee that revoked her tenure was supposed to address. The question the Hearing Committee was supposed to address, the committee that revoked her tenure, was to be whether the business school had established, with clear and convincing evidence, that she was guilty of academic fraud.<\/p>\n<p class=\"para\">This is a point that\u2019s often lost on people who are blessed not to be lawyers. So, with apologies to the less blessed, let me unpack a little bit what that qualification actually means.<\/p>\n<p class=\"para\">If you drive your car and you negligently sideswipe another car, and that other person sues you, demanding you pay for the damage you\u2019ve caused, the question the fact finder, typically a judge, sometimes a jury, would answer is this: is it more likely than not that you, the defendant, drove negligently. That question is asking a simple probability question. Is the fact finder 51% confident or more that you were negligent? If it is, you are guilty. If it is not, you are free.<\/p>\n<p class=\"para\">By contrast, if you are accused of a crime, and God forbid, you are ever accused of a crime in the American criminal justice system, because that system is just a total disaster&#8230; But if you are, and you do what literally 98% of federal criminal defendants don\u2019t do, you decide to go to trial, then the question the jury at that trial will have to address is whether there is proof beyond a reasonable doubt that you are guilty.<\/p>\n<p class=\"para\">The law doesn\u2019t translate that standard directly into probabilities. It\u2019s more like a confidence that you must have. So, if you\u2019re a juror on a criminal trial, you should feel 90 to 95% confident that the person is guilty. That\u2019s an extremely high level of confidence, but it is the level that we, as a society, have decided is appropriate to avoid the horrendous outcome that the innocent would be convicted wrongly. Though we can be 100% certain, there are plenty of innocent people who are convicted wrongly.<\/p>\n<p class=\"para\">The standard the Hearing Committee was supposed to apply in deciding whether to terminate Francesca\u2019s academic career stands between these two standards. The standard was clear and convincing evidence, which the Supreme Court has explained as &#8220;a firm belief or a conviction of guilt.&#8221; And as scholars and courts have translated it, they\u2019ve said it\u2019s a 70 to 80% level of confidence. Some have said even higher, 75 to 85% level of confidence.<\/p>\n<p class=\"para\">I talked a bit about this in the third episode. Now, if you\u2019re like me or someone like me, and you believe that Francesca is actually innocent, the tragedy of this story is no matter what happens now, a significant proportion of people will think that she\u2019s guilty.<\/p>\n<p class=\"para\">Compare the story of that of Amanda Knox, the American student in Italy who was accused of murdering a close friend, but then, years later, four years later, was found not guilty. Many still believe she is guilty, even though a man was convicted of the crime based on DNA evidence that linked him directly to the crime scene, while there was no DNA evidence tying Knox to the crime scene, not to mention the fact that she had zero motive to murder one of her closest friends. Still though, Knox must live in a world where many think her guilty, because, after all, that\u2019s what the courts originally concluded.<\/p>\n<p class=\"para\">And indeed, that\u2019s perhaps the most common reaction I\u2019ve gotten as I\u2019ve become public about defending Francesca. How can you be so sure, Lessig, when there was such an extensive process undertaken to determine whether she was guilty? The business school ran an extensive process. The university ran an extensive process. Literally millions of dollars have been spent to determine whether Francesca was guilty, and they concluded she was.<\/p>\n<p class=\"para\">When someone says something like that to me, I just want to say, grow the fuck up. Actually, I\u2019m sorry, kids, that\u2019s not an appropriate word to use unless it is in an appropriate context which this plainly is. This naive, childlike belief in process, in legal process, is just astonishing to me.<\/p>\n<p class=\"para\">The unwillingness to stand back and be critical is a weakness, not a virtue. The failure to recognize the inertia the bureaucracies unleash is an obliviousness that I cannot believe intelligent people entertain.<\/p>\n<p class=\"para\">Again, the Amanda Knox case is a sad parallel, a kind of inverse of Francesca\u2019s case. There, an American was swept up into a Byzantine Italian legal process that could not find an obvious truth despite years of judicial process. She sat in jail for four years.<\/p>\n<p class=\"para\">In this case, there\u2019s an Italian swept up into an American, Byzantine legal process that refused to acknowledge the flaws, the obvious flaws, at the very beginning of this case, and see how those flaws would tilt the whole process against its possibility of determining any truth. As the months of litigation continued, as the cost of that litigation mounted at each stage, if you know anything about the psychology of people in such a process, you know that these people were thinking to themselves, &#8220;geez, the conclusions below just have to be right, or I need to be absolutely certain that the conclusions below are not right if I\u2019m going to reverse them now.&#8221;<\/p>\n<p class=\"para\">If the committee, the Hearing Committee, was actually an appellate court, a court whose job it was to review the fact finding of a lower court, there might be some excuse for that perspective, because ordinarily, an appellate court reviewing the fact finding of a jury is not allowed to decide whether it believes the jury was right or wrong. It\u2019s supposed to decide whether it thinks that the decision of the jury was clearly erroneous, and only if it was clearly erroneous do they have the right to reverse the finding of a jury.<\/p>\n<p class=\"para\">Indeed, I\u2019m sure that\u2019s the hardest part about of being an appellate judge, that they have to review these cases that they are certain were wrongly decided, but they\u2019re not allowed under the rules of appellate procedure to reverse because the mistake was not clearly erroneous.<\/p>\n<p class=\"para\">But in this case, the Hearing Committee was not reviewing the fact finding of the business school under a rule of deference. Indeed, the Hearing Committee expressly said, &#8220;The Hearing Committee intends to review and consider the report of the Harvard Business School Investigation Committee as one part of the evidentiary record for this matter. As discussed below, the Hearing Committee will also consider any response provided by Professor Gino to that report. In addition, the Hearing Committee will conduct hearings and make findings of fact as required by the rules and governing Third Statute proceedings.&#8221;<\/p>\n<p class=\"para\">The committee\u2019s job, as the committee itself believed, was to review the evidence de novo, meaning literally review the evidence anew, and decide whether that evidence established under a clear and convincing standard that Francesca was guilty.<\/p>\n<p class=\"para\">And I get it. There\u2019s lots in this case that\u2019s hard. There are plenty of conflicting facts. There\u2019s a lot of hypotheticals about what could have happened, and lots of daydreaming by the most cynical, that maybe Francesca is some sort of evil genius, and that she did everything she did in order to both cheat and to cover up her cheating. There\u2019s tons of speculation.<\/p>\n<p class=\"para\">But here is the easiest fact in this case. You cannot look at this evidence and draw the conclusion that there is clear and convincing evidence that she committed academic fraud.<\/p>\n<p class=\"para\">Again, I think she\u2019s absolutely innocent. But the point is, the critical point is, the reason why what the Hearing Committee did was wrong is, whatever the uncertainty, there is no uncertainty about this, the evidence, this evidence does not meet the standard the University has established for convicting somebody of academic fraud.<\/p>\n<p class=\"para\">Believe what you want about whether she\u2019s guilty, the University is plainly guilty for failing to live up to its own standard. It has, for the first time in its 390-year history removed the tenure of a faculty member by misapplying the standard established by the University to protect the tenure of faculty members.<\/p>\n<p class=\"para\">Now that\u2019s not to say this is the first time Harvard has forced a faculty member out. There are lots of cases where somebody was accused of wrongdoing and the University succeeded in forcing them to step aside. Some of those cases I know pretty well, and with the ones I do know, I\u2019m glad the University succeeded in getting the faculty member to step aside.<\/p>\n<p class=\"para\">But this faculty member, Francesca, when accused of a crime, said, &#8220;Hell no, I did not do it. And I\u2019m asking you to apply your standard to determine whether I can keep my job despite the slanders against me.&#8221; She fought back.<\/p>\n<p class=\"para\">And it is my firm belief that if the University had applied its standard properly, she would have won. That\u2019s not to say she would have exonerated herself, at least in the eyes of a distracted public. The charge against her will always stain her reputation, however wrongfully, and I believe absolutely wrongfully it was that they were raised against her.<\/p>\n<p class=\"para\">But this incident also stains Harvard\u2019s reputation.<\/p>\n<p class=\"para\">And I should think at least among Harvard faculty who work with data, it should create a certain chill, because are you absolutely confident that in the processing of your data by the RAs or lab assistants who actually handled your data, there were not mistakes made? Mistakes that might suggest that you had fraudulently represented your results? Because if you are not absolutely certain that such mistakes were not made, then I would strongly recommend you stop citing those papers and pray that within the next six years, no one raises any questions about them. That\u2019s the reality that follows from the prosecution of Francesca Gino. That reality is just absurd.<\/p>\n<p class=\"para\">Okay, now look, I get it. There are a lot of people who will cheer the fact that a tenured professor at Harvard, meaning me, has charged Harvard with a great wrong, because there are many people who hate my University.<\/p>\n<p class=\"para\">But, as I said at the start of the season, I am not among the people who hate Harvard University. I love this university, and I\u2019m among the people who would die to defend the academic freedom that universities like Harvard and Harvard permit.<\/p>\n<p class=\"para\">That freedom is this podcast. I\u2019m criticizing something I love because it has acted wrongly and in its wrong it has done enormous and unjust harm to a decent and brilliant scholar.<\/p>\n<p class=\"para\">It is the greatest honor of my career that I get to work at an institution like this. It is an even greater honor that I live within a culture that permits me to criticize openly and freely, an institution that I love, because to criticize is not to condemn. To criticize is the opening move in an o\ufb00er to repair.<\/p>\n<p class=\"para\">Okay, so one final point before we turn to the substance of allegation number four.<\/p>\n<p class=\"para\">In the time between the last episode and this episode, Wall Street activist investor Bill Ackman revealed that he has been supporting Francesca in her fight to clear her name. In a post on X, which has received millions of views, Mr. Ackman described why he had concluded that the charges against Francesca were false, and he indicated that he would support her in her e\ufb00ort to defend her name and reputation.<\/p>\n<p class=\"para\">So, there\u2019s obviously a backstory here. The brief version is this: when Francesca began to run out of money to pay her lawyers to defend herself, I think she\u2019d spent over a million dollars at that point and was drawing on her retirement and her children\u2019s education fund, she realized, and her friends realized that she needed outside support if she was going to continue. I then reached out on her behalf to a number of people who Francesca knew, who might be in a position to o\ufb00er that support.<\/p>\n<p class=\"para\">Bill Ackman was one of those people. He responded and wanted to hear the story directly from her. So Francesca, Mr. Ackman and I and one other person from his team listened to Francesca describe her story. Bill Ackman asked her questions, probing questions. He then took a significant amount of time to work with his sta\ufb00 to fill in the details so that they could come to believe that his initial reaction, his initial instinct was correct. Those instincts were the same as his conclusion that she had been wrongly convicted.<\/p>\n<p class=\"para\">And so, for the last couple of years, he has been the critical behind the scenes financial support that has made it possible for her to defend herself against Harvard. But for his support, she would be personally bankrupt, without a job, and with no chance of restoring any part of her reputation.<\/p>\n<p class=\"para\">Now, seeing the story about Bill Ackman supporting Francesca develop has been a little surreal for me. I try hard to avoid things like this on social media, but friends sent me a couple of choice tidbits. There was outrage out there among Trump supporters who were criticizing Bill Ackman, himself a supporter of Donald Trump, for his backing a project supported by somebody like me, an opponent of Donald Trump. What was his problem? They asked. How could he support something that a critic of the President was also supporting?<\/p>\n<p class=\"para\">I don\u2019t know Bill Ackman personally. I have enormous respect for the particular genius that would allow him to be so successful. I was also incredibly impressed when he came out as a strong supporter of birthright funds, which would be a government-funded investment accounts for every child, giving them a lump sum that is invested immediately and broad, low-cost equity funds held in tax free accounts until their retirement. (That\u2019s close to maybe better than an idea my mentor, Bruce Ackerman has championed with Anne Alstott in their book The Stakeholder Society. Bruce and Anne are also not Trump supporters, proving again that ideas can be thicker than politics.)<\/p>\n<p class=\"para\">There are many things that Bill Ackman has said that I support. There are many things he has said that I don\u2019t support. But so what?<\/p>\n<p class=\"para\">I consider it a strength that even someone with whom I disagree could see the case as I do. That is, that the case against Francesca is fatally flawed. And I can\u2019t express enough my gratitude that he has been willing to give her the chance to prove it.<\/p>\n<p class=\"para\">Okay. Finally, let\u2019s get back to the promise of this episode: Allegation number four.<\/p>\n<p class=\"para\">Once again, I\u2019m joined by an interlocutor. Like last time, the words have been written by a friend who is expert in data analysis. Like last time, for complicated reasons that continue still, that person can\u2019t talk freely about the case. But this time, they\u2019ve recorded the podcast with me, and I\u2019ve used fancy AI from Eleven Labs to replace their voice with an AI\u2019s voice.<\/p>\n<p class=\"para\">I\u2019m going to call this AI Ava. Ava and I will go back and forth discussing this allegation, just like I did with Ron Susskind and allegation number two.<\/p>\n<p class=\"para\">The reason for this style is the reality: that humans are pretty good at understanding conversations, and they get distracted listening to monologs, especially monologs with beautifully mellifluous voices of someone like me. Seriously, how many times did you check your Instagram during the last rant I just unleashed? See? See my point?<\/p>\n<p class=\"para\">Okay, so that\u2019s the introduction. Let\u2019s turn now to the episode.<\/p>\n<p class=\"para\">So welcome, Ava. Tell us a little bit about allegation number four.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Hi, Professor. Thanks for having me. Happy to help unpack this story to make the weakness with allegation four clear.<\/p>\n<p class=\"para\">This allegation involved a study that Francesca conducted with four other researchers over 15 years ago while she was at the University of North Carolina.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">15 years ago?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">That\u2019s right, 15 years ago.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">I don\u2019t understand. Doesn\u2019t Harvard have a policy that says that allegations of academic misconduct more than six years old, &#8220;may not be investigated&#8221;?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Yeah, that\u2019s a rhetorical question, right? I mean, you covered that. Was it in episode two?<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">It was.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">So yes, Harvard has a policy consistent with regulations from the US Department of Health and Human Services, O\ufb03ce of Research Integrity, which recognizes a six-year limitation period on allegations of misconduct due to &#8220;the problems that may occur in investigating older allegations and the potential unfairness to the respondent in defending against them.&#8221;<\/p>\n<p class=\"para\">But Harvard believed there is an exception to that policy if the alleged fraudulently created data continued to be &#8220;used in some particular way.&#8221;<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">&#8220;Used in some particular way.&#8221; What does that mean?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">In episode three, at the transcript beginning at page 27, you explained the policy. The tl;dr is this: the statute of limitations was originally a bit ambiguous, and some people thought merely citing an earlier work was enough to expose the author to subsequent investigation.<\/p>\n<p class=\"para\">That way of reading the rule, however, would make the protection of the rule meaningless. Merely posting a list of your works, like a CV or on a website, would mean that all your works were continually subject to investigation. I don\u2019t agree with that reading, but to remove any doubt, in 2024 the rule was explicitly modified.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Modified how?, he asks, trying to sound like this, too, is a genuine question.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Modified to indicate that the class of cases they\u2019re exempting from the &#8220;no investigation rule&#8221; is when the earlier research is being relied upon in subsequent research.<\/p>\n<p class=\"para\">The precise language of the modification is &#8220;citation to the portion, or portions of the research record alleged to have been fabricated for the potential benefit of the respondent.&#8221;<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Portions of the research record seems clearly to signal something more than a mere citation, right?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Right. It seems to comport with what makes sense: that if you rely on the allegedly fabricated data in subsequent work, you lose the benefit of limitation, but not if you simply cite an article alleged to have fabricated data behind it.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So how precisely did Francesca cite the papers here?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">The short answer is simple and sweet. She never cited the research to support or rely on the findings subject to the allegations. She simply cited the papers.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay. So, to underline this point, she never cited the work in a way that should trigger the exception to the rule barring investigation, so that the rule barring investigation should have barred the investigation of this allegation, right?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Exactly.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay. Again, so with the law professor\u2019s obsession about process, please recognize how incredibly important this process limitation is.<\/p>\n<p class=\"para\">If you believe in what the law calls statutes of limitations, if you believe in the justice of a statute of limitation, which is not about allowing guilty people to go free, but about allowing innocent people the freedom to live their lives without fear that they will be wrongfully accused of crimes allegedly committed in this case, 15 years ago, then please recognize how wrong it is to allow the prosecution here of this 15 year old charge.<\/p>\n<p class=\"para\">Most of the records surrounding the facts in this case just do not exist. Practically no one would have any recollection about what actually happened on the days allegedly constituting the crime. Email records don\u2019t exist. Intermediate copies of files don\u2019t exist. The IRB determination does not exist.<\/p>\n<p class=\"para\">Bottom line, this is a radically incomplete record. And thus, on the basis of this radically incomplete record, it is wholly improper for the University to charge someone with a crime. Even if the evidence were absolute and overwhelming, it would be wrong to prosecute this crime. That is what due process means.<\/p>\n<p class=\"para\">But in this case, the evidence is not absolute and overwhelming. It is not even persuasive. And yet, Harvard prosecuted this alleged crime against Francesca. It is just wrong.<\/p>\n<p class=\"para\">Okay, yeah, so that\u2019s another rant. And I\u2019m sorry, let\u2019s get back to the actual allegation here.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Absolutely.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So, tell us what happened.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Francesca and her co-authors wanted to do a study to evaluate the e\ufb00ect of a pledge of honesty on someone\u2019s actual honesty.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">A pledge of honesty?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Yeah, you know, like a document you sign that says, \u2018I promise to be honest.\u2019<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Do people really sign such documents?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Of course. For example, when a student takes an exam, often they sign an honor pledge promising that they won\u2019t cheat, or when you promise to tell the truth before you\u2019re deposed, or when you submit your taxes. There are examples of pledges of honesty all over the place.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so how did they intend to measure whether a pledge of honesty actually a\ufb00ected actual honesty?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">The way it was to work was this. And remember, this is 15 years ago, so it was all in real space, in real rooms, in real buildings at the university Francesca there was, at the University of North Carolina.<\/p>\n<p class=\"para\">And most astonishingly, it was all done on paper. These were not study participants giving answers on the computer screen. These were subjects filling out answers to real questions in real rooms on paper. So the real space in particular was two rooms. In one room, the subjects were given a bunch of math problems to solve under time pressure.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Math problems?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Math problems, love it. The whole exercise was framed around completing math problems and being paid for their performance on those problems. Participants were going to be paid a higher<\/p>\n<p class=\"para\">amount than what people usually received as a payment in a regular study, because, they were told, they would be taxed on their earnings.<\/p>\n<p class=\"para\">And so, the question was whether they over reported their performance on the math problems they solved in the first room when filling out a form that looked like a tax form in the second room.<\/p>\n<p class=\"para\">Here\u2019s the important point: after they solved the math problems, they scored their own answers.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Why would you let them score their own math problems?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">It was all a setup so that they would have the opportunity to cheat.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Ah. So, they do the math problems in room one, score their answers. They then go to room two and report the number they got right on this tax form.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Right. And on some of the tax forms, there was a pledge of honesty.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Sounds almost like Monty Hall. Okay, so what precisely happened then in this room two.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">In room two, there were basically three di\ufb00erent treatments.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Treatments, what is this? A spa now?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">No. Treatments is the term in social science for the di\ufb00erent processes that di\ufb00erent subjects receive in an experiment. The whole purpose of the research is to compare how these di\ufb00erent &#8220;treatments&#8221; or processes a\ufb00ect the outcome. By comparing these di\ufb00erences, researchers hope to make inferences about the relationship between treatment and outcome, between cause and e\ufb00ect.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so what were the treatments these students were subjected to?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Subjected to is a little harsh, but okay\u2026 In all three treatments, the students were going to report how many of the math problems they got correct, and they did so on a form that looks like a tax form.<\/p>\n<p class=\"para\">In one treatment, before the students reported, they signed a pledge of honesty, saying, &#8220;I declare that I will carefully examine this return and that to the best of my knowledge and belief it is correct and complete.&#8221; That language is similar to what you a\ufb03rm on a real tax form.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay. So, in this treatment, they sign a document that basically says, \u2018I\u2019m going to be honest.\u2019 Then they answer the questions about how many answers they got right in room one. And the study then tracks whether they, in fact, were honest.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Right.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">How do they know whether they\u2019re being honest?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Well, the participants didn\u2019t know this, but although they didn\u2019t sign the sheets on which they worked out the answers to the math problems, there was a unique identifier for each participant on those sheets. The RAs conducting the research in room one collected the answers and could thus tell the number of math problems that each participant actually got correct.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">But what\u2019s the incentive to lie? Do they get paid money based on how many they get right or get wrong?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Yes, they get paid more the more they get right.<\/p>\n<p class=\"para\">Okay, so in treatment one, the subject signs a form that says \u2018I\u2019m going to be honest,\u2019 and then reports the number of math problems they got right. The researcher had a trick to determine whether they were exaggerating, and if so, by how much.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so what\u2019s treatment number two?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">In Treatment two, they first report the number of answers they got correct, and then they sign the tax form at the bottom. That form at the bottom includes a pledge of honesty.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay. So after they have written down the number of questions they got right, they then pledge to be honest. It\u2019s kind of hard to imagine how that pledge could a\ufb00ect the number they had already reported, at least assuming that time travel is not possible.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Well, there are actually plenty of forms like this. On the US tax return, for example, you first complete your tax return and then you sign at the bottom. So someone clearly thinks it works to have people promise honesty at the end of a process. Whoever designed the IRS forms must think that. But yeah, it\u2019s a little weird. From first principles it\u2019s not obvious why this would work.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so then what was treatment number three?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Treatment number three is the control case. There is no signature space on the tax form, neither before they complete the form, nor after. The participants are simply asked how many math problems they got correct.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so after they\u2019ve collected all these data and analyzed it, what did the authors conclude?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">They concluded that, in fact, signing the form, saying that you were going to be honest, that is signing at the top of the form, had a statistically significant e\ufb00ect on cheating. When they signed first, participants were, in fact, more honest than when they signed after or did not provide a signature at all.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so then what was Harvard\u2019s allegation about this research?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Well, the allegation is divided into two parts. For now, let\u2019s stick with the part related to the data used in the study.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And what was the allegation about that data?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">It basically came down to the di\ufb00erence between two files. The first was the dataset Francesca analyzed and posted online on OSF. She shared the write up of the results from that data set on July 18, 2010, with her co-authors. Call that File B. The second file is a different data file that was sent to Francesca by her lab manager on July 16, 2010. Call that File A. HBS data consultant Maidstone found di\ufb00erences between these two files. Those di\ufb00erences were the foundation for the claim that Francesca had modified the data fraudulently.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So what do you mean by di\ufb00erences here?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Well, the values in some cells are di\ufb00erent. Specifically, there are 73 di\ufb00erences, plus three new rows of data.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And so why would that di\ufb00erence be significant?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">The suggestion is that the di\ufb00erences in the file, all but one of which strengthen the conclusions of the paper, must be di\ufb00erences that were introduced by Francesca. And if she introduced those di\ufb00erences to strengthen the conclusions of her paper, well, that\u2019s fraud.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, that\u2019s agreed. But let\u2019s be sure we understand the context for these data files. So, who produced these files?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">As you described in episodes two and three and a bit in four, the production of the data, which the academics then evaluate, is a process that is conducted by others: by graduate students, undergraduates and sometimes people hired as lab managers or research assistants (RAs). These are people who find the academic work interesting, or maybe just need a job. Some may want to become academics themselves. They help conduct the experiments and prepare the data to be analyzed.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And, again, to repeat a little bit what we said before or what was said earlier, in earlier episodes, what do you mean by prepare the data?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">The data has to be, researchers call it, cleaned. In this case, first, it had to be typed into Excel because it originated on paper forms. Then, within the data files, if there were any inconsistencies, they had to be flagged or corrected or somehow dealt with. Then the data needed to be put into a format that would allow for statistical analyses.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And, again, to beat a dead horse: who is doing this cleaning process?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Basically, anybody except the professors running the ultimate analysis. I mean, the professors could do it, but they always want RAs or students to do it for them to make better use of their own time. Plus, the people who actually ran the experiment would know best who to exclude from the data set. They would see who did not follow instructions or was clearly not paying attention.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">But in this case, who specifically conducted that data work?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">In this case, the data work was done by Francesca\u2019s lab manager at UNC, Jennifer Fink, and one or more RAs who helped her run the study. In its investigation, HBS contacted Jennifer. She shared the three files that she still had. Each of them was attached to emails she had sent to Francesca back in July 2010. Each of them had the same name, but had been saved at di\ufb00erent times. The last one to be saved for this study is the one we\u2019re calling File A. HBS concluded that File A was the final file given to Francesca before she performed the analysis for the paper.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so the lab manager was running the lab at UNC, and she ran this experiment. She and some of the RAs perhaps, gathered the data, transcribed it, cleaned it, prepared it, and gave it to Francesca.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Right. And the allegation here is that there is a di\ufb00erence between the data they gave to Francesca finally, supposedly File A, and the data that Francesca used as the basis for the analysis, File B. In the language that you\u2019ve used in the course of this podcast, we can call those di\ufb00erences the anomalies. The question is: who produced those anomalies? The Hearing Committee believed that the time between the last time File A was saved and the time File B was created makes it di\ufb03cult to imagine that those anomalies were produced by anyone other than Francesca. So, they concluded she must have committed fraud in producing File B.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, but that assumption only makes sense if File A actually is the final work product of these research assistants or the lab manager, right?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Right. The assumption of the Hearing Committee is that File A represents the final file that Jennifer and the research assistants worked on. That File B was not worked on by Jennifer and the research assistants. File B has anomalies within it, the di\ufb00erences between it and File A. The Hearing Committee concluded Francesca produced those di\ufb00erences.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so that\u2019s clear. So, File A and File B are di\ufb00erent. The di\ufb00erences, except in one case, strengthen the conclusions of the paper. To harp on a point I made in an earlier episode, and we\u2019ll make even more strongly in the next two episodes, not all the changes actually strengthen the conclusions of the paper. But the di\ufb00erences in general strengthen the conclusions here.<\/p>\n<p class=\"para\">Like the HBS Investigation Committee, the Hearing Committee believed, therefore, that there was motive and opportunity, and that therefore Francesca was guilty.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">That\u2019s what they concluded.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Right, we don\u2019t conclude that. I don\u2019t conclude that. I don\u2019t believe she\u2019s guilty at all. I continue to believe, as she insists, that she is innocent, but it doesn\u2019t look good, right?<\/p>\n<p class=\"para\">I mean, there are di\ufb00erences between File A and File B, with one exception, those di\ufb00erences do strengthen the conclusions of the paper. Francesca was the one working with File B, so it certainly sounds like she must have made those changes.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Yeah, it looks like that, except for one important fact about the context, which you were just hinting at. We do not know that File A is actually the final file as prepared by Jennifer and the research assistants and given to Francesca to analyze.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n`<p class=\"para\">Okay, so I\u2019ll play along. What do you mean by that, Ava?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Well, File A is a file. It\u2019s produced after earlier versions of the file, but it doesn\u2019t announce itself as the final version of the file. How could it? We\u2019ve all seen files with names like Final or final final or revised, where someone thought a final a file was final, but then ended up making additional modifications. Anyway, there\u2019s not even a final name like that here. The actual name of the file was &#8220;Taxstudy.&#8221;<\/p>\n<p class=\"para\">And it\u2019s not like there is an o\ufb03cial archive in which each version of this file was stored. We\u2019re talking about some email attachments.<\/p>\n<p class=\"para\">What we know for sure is this is one version in the life of this data, a life that began when the data was transcribed from paper forms that no longer exist. But we don\u2019t know whether there were other modifications made to this file before it was finally given to Francesca.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Finally?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Right, finally. There\u2019s evidence she got a version of File A on July 16th, but we don\u2019t know whether there were other versions of file a that she got later than July 16.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so what you\u2019re saying is that File A is a file, but there\u2019s nothing to indicate that it\u2019s the final version of the file that was given to Francesca for her to evaluate.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Exactly. Jennifer did not testify that File A was the final file. The Investigation Committee did not ask Jennifer whether she was certain that File A matched the raw data, nor did Jennifer say it matched the raw data. And HBS own data consultant said it could not be sure, either, as it did not have access to the raw data set or the complete email records for the time. It was HBS that concluded that File A was the final data file given to Francesca to analyze.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Did anybody testify that File A was the final data file?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">No. Nobody testified that File A was the final file. There was no actual evidence from witnesses that File A was the final file given to Francesca before it was analyzed by her to produce the paper.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Did Harvard check Francesca\u2019s email?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Not surprisingly, she doesn\u2019t have her email from 15 years ago from a di\ufb00erent university and a di\ufb00erent computer. In June 2022, Francesca tried to obtain her email records from UNC so she could reconstruct the research process related to the study. UNC informed her that, since she was no longer employed, she wouldn\u2019t have access. And so she was going to go talk to the Dean at UNC to make the case, but UNC IT told her that it was pointless, because their email was only retained for five years anyway.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so Jennifer provided three versions of this file we\u2019ve called File A, but is there any indication that there were no other files?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">No, and that\u2019s the point. There is certainly no ledger of what files previously existed a dozen years after people began looking and no other files were found. But that really doesn\u2019t prove that there weren\u2019t other files at the time.<\/p>\n<p class=\"para\">And here\u2019s where time is so incredibly important, the evidence of files shared on email a dozen years ago is likely radically incomplete. Francesca testified that much of her work back then, continuing through the pandemic, was happening through exchange of data on USB thumb drives at in person meetings, not across email. In July 2010, Francesca was still at UNC ready for a move to Boston later that month. There in person, it would have been natural for her to continue to meet with Jennifer in person and exchange files using USB keys. But of course, no one has 15-year-old thumb drives.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So there\u2019s at least a presumption, right, that File A is the final file, because no other file later than File A but before File B has been found. But is there anything in the content of File A to suggest that it actually wasn\u2019t the final version of the file. Anything to rebut this presumption that it was the final file?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Absolutely. In the email with an earlier version of File A, Jennifer indicated that there were problems with the data as she wrote &#8220;the people are SERIOUS dumb, dumbs on this study. They seem to be having some serious issues calculating the money, or if they got the amounts right, they were written and scribbled in very strange ways on the form.&#8221;<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">When was that email written?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">It was written after the earliest version of File A that we have was produced, but before File A was produced.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And did the final version of File A correct the problems that Jennifer\u2019s email was referring to?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">It did not. File A still needed to have that work done. We just don\u2019t have a version of File A with that work done. Or put di\ufb00erently, we don\u2019t have a version of File A where the work that Jennifer had identified needed to be done was actually done.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so that strongly suggests there was another file.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Bingo. Jennifer is indicating that she has work that she needs to do, File A does not include that work. All indications are that she was a great lab manager, so having flagged work that needed to be done, surely she would have done it. We just don\u2019t have the file that demonstrates that she did the work, or that shows data after she did the work.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Which would mean we don\u2019t have the final version of File A, correct?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Correct.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so is there other evidence to suggest that File A is not the final file?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Yeah, sorry to bury the lead. Here\u2019s the most conclusive evidence that file a is not the final file. It doesn\u2019t even include all the participants in the study.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Wait, it\u2019s incomplete. It doesn\u2019t include all of the participants in the study. How do we know that?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Well, because it turns out, a dozen years after the fact, Francesca still had paper receipts for the payments made to these participants. It makes sense that she would have those receipts, at least initially after a move to Boston. Her HBS job started on July 1, 2010. She moved to Boston in late July, she needed the physical receipts so that HBS could reimburse her for the study she had conducted. That\u2019s why she would have brought them to Cambridge.<\/p>\n<p class=\"para\">But late in the summer of 2023 she discovered she had never thrown them away when she unpacked some boxes in her garage, boxes delivered by HBS from her o\ufb03ce after they put her on unpaid leave, she found a box with the receipts. Those receipts demonstrate conclusively that there were more participants who completed the study than were reported in File A.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Oh wait, so what you\u2019re saying is that she has paper receipts demonstrating that there were more people who completed the survey than are reported in File A, meaning obviously File A is not the final data file for this study. Are those people included in File B?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Yes. By examining the paper receipts for the study, Francesca determined that file a does not match the paper receipts while File B does. This suggests that the committee did not have the final file that Francesca was working from. They therefore did not have the predicate for saying she changed anything.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Slow down. So, what do you mean by matching here?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">When subjects were paid for their participation in studies at UNC, the RA running the study, recorded the payment on a payment record with subject name, address, ID, number and amount paid. From the subject name, it is possible to determine gender because the name is common, or identify it via online searches, for example, LinkedIn.<\/p>\n<p class=\"para\">Meanwhile, the data file can be used to calculate the amount paid based on participants answers, and the data file indicates the gender of each participant. So there are two fields to match: gender and payment amount.<\/p>\n<p class=\"para\">The receipts Francesca found have records for 310 participants. The study at issue only had 101 participants. The 209 additional participants relate to other studies running in the same UNC lab in the same period. By coding participant gender from names on payment records and matching payment amounts in gender with what\u2019s in the data file, you can see that all the payments listed in File B are accounted for in the receipt records.<\/p>\n<p class=\"para\">By contrast, File A showed discrepancies. For example, file a listed more participants at certain payment amounts than were actually paid according to receipts. Specifically, according to File A, three participants should have received $7 but the subject receipts indicate that only two subjects were paid $7. Similarly, File A says 14 subjects were paid $16 but the payment receipts say only 11 were paid that amount. By contrast, File B matches the paid subject numbers, and that strongly supports the conclusion that File B is actually the final data that was given to Francesca after correction of the errors in File A by the people who corrected data, RAs in general, or Jennifer in particular.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so Francesca\u2019s lawyers must have presented this obviously exonerating evidence to the Hearing Committee, right? I mean, how did the Hearing Committee address this evidence?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">This is one of the many examples of the Hearing Committee revealing that they didn\u2019t understand what was in their record. In response to her lawyers pointing to these receipts, the Hearing Committee wrote, &#8220;Professor Gino claims to have reviewed the original paper receipts completed by study participants and verified that the later data on which her analysis relied are accurate. She did not, however, provide those receipts or explain how they account for the analysis data set.&#8221; (And the citation there is A650) That sounds bad, but it\u2019s flatly contradicted by the actual record, because those receipts were, in fact, in the record at RX 626A, at 1708 to 10, and RX 626B. Maybe we can post all that to the website, blurring the names.<\/p>\n<p class=\"para\">Yeah, we\u2019ll do that for sure. But this is really, truly astonishing. I mean, I met and worked a bit with Francesca\u2019s lawyers during the tenure revocation hearing. They were superstar lawyers, incredibly competent, and they conveyed their competence in every filing and every minute of their oral advocacy.<\/p>\n<p class=\"para\">So, imagine the chutzpah of a committee that is told by the lawyers, look there are receipts, and yet doesn\u2019t recognize that if the lawyers say there are receipts, there are freaking receipts. The lawyers are not going to lie about something like that. They\u2019re not going to say that there are receipts when there are not receipts.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So, you would think that the Hearing Committee would think to itself, geez, have we missed that there are, in fact, receipts? Can someone go and find the receipts in the record? But that\u2019s not what they did. Instead, they just assumed, oh, here yet again: Francesca must be lying.<\/p>\n<p class=\"para\">But no, Francesca was not lying. Her lawyers were not lying. She was honestly reporting that she had found the receipts and that the receipts did show that there were people who participated in the study who were not reflected in File A. Which means, again, File A was not the final file given to Francesca for her and her colleagues to evaluate.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Yeah, that\u2019s right. I wish I had been in the Third Statute hearing room, or in the room where someone, I guess, lawyers, drafted the remark that Francesca didn\u2019t provide the receipts. Because she plainly did. And those receipts plainly showed that the whole predicate to the committee\u2019s conclusion that she changed the values, that just disappears. The only file that\u2019s in the record that includes all the participants is File B.<\/p>\n<p class=\"para\">Okay, so let\u2019s pull this together.<\/p>\n<p class=\"para\">The whole of this charge against Francesca with respect to the data is the di\ufb00erence between File A and File B. But for that charge to be a valid charge, the committee had to establish that File A was, in fact, the final file as completed by her research lab manager, Jennifer, and that it was the file from which Francesca did her work.<\/p>\n<p class=\"para\">But in fact, there\u2019s plenty of evidence that it wasn\u2019t the final file. Most conclusively, that the paper receipts demonstrated that that file did not include all the people who actually took the survey. The only way to conclude that she changed the data is to provide the baseline from which the data was changed. Without that, there is no foundation for the claim Francesca modified the original data. The only file with all the data is the file that was actually posted on the OSF site.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">That\u2019s right. The whole foundation for a charge of academic fraud is that she changed the data in a way that strengthened her conclusion. But the foundation for that claim does not exist.<\/p>\n<p class=\"para\">It\u2019s not surprising that it doesn\u2019t exist. Everything happened 15 years ago. But that\u2019s just another reason why it\u2019s improper to prosecute a claim of fraud that\u2019s 15 years old.<\/p>\n<p class=\"para\">It\u2019s not just that the charge is not timely. It\u2019s also that the evidence is so deeply flawed by the incompleteness of the record, that there\u2019s no fair basis on which one could conclude that she\u2019s guilty. Again, the only file in this record with all of the participants in it is the file that was posted online, File B.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, but this story just gets worse, because then there\u2019s another allegation the Hearing Committee took up with respect to this paper. Tell us about the other part of this allegation, which we could call Allegation 4A.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Yes, this part is even crazier. The essence of the charge is that Francesca initially described and conducted one experiment, but when a co-author later pointed out that that experiment didn\u2019t make sense, Francesca changed the description to describe a di\ufb00erent experiment.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so you\u2019re going to have to unpack this a bit. What exactly does that mean?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Well, remember, Francesca was initially working with two other academics in designing this study. They joined forces with two other researchers who had field data that would make the paper stronger. When the two note newcomers read the details of the July 2010 study in 2011, one of them pointed out a potential issue. Francesca made edits to respond to the issue.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And so what was the issue?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">The original write-up described a study as requiring the subjects to do math problems in room one, being paid in room one, and then in the second room, making a pledge about honesty or not, and being paid again based on the results in room two.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And so, wait. Why would they be paid in the first room if the purpose of the study was to measure whether they are honest and reporting their results, and that\u2019s done in the second room? And why pay them twice?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Well, exactly, it would make no sense. If the payment occurred in both rooms, that would imply that the dependent variable, the thing they were trying to measure here, whether the subject lied, was completed before the independent variable whether or when they made a pledge. But that\u2019s backwards.<\/p>\n<p class=\"para\">It could only make sense to alter the independent variable and then see how it a\ufb00ects the dependent variable. Yet, the original draft of the paper seems to suggest that was the plan. It wasn\u2019t, and it wasn\u2019t what actually happened.<\/p>\n<p class=\"para\">So, when one co-author noticed the weirdness in the description and raised it to Francesca. Francesca fixed it by conforming the description to what actually happened.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Conformed it how?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">By changing the description of the study, explaining that payments were made in the second room only.<\/p>\n<p class=\"para\">But Harvard saw this change and charged that this change was deceptive. That in fact, the study had been run in the flawed, braindead way described in the earlier draft, and that Francesca covered up that blunder by revising the draft rather than discarding the flawed data. That cover up, they alleged, was academic misconduct.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, but what evidence is there that the study was not conducted as the original version of the paper was described?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">With one exception that I guess we\u2019ll take up in a minute, all the rest of the evidence that there is in this case. First, and again, simple logic, the design would be brain dead. No one, certainly not these academics, would make such a stupid design choice.<\/p>\n<p class=\"para\">Second, the lab manager, Jennifer, who was actually in the room, or rooms, where it happened. She did not say that there was a payment in the first room. In fact, she observed that if there had been a payment in the first room, then in some cases, they would have had to take money back from participants in the second room. She said she didn\u2019t remember taking back any money at all.<\/p>\n<p class=\"para\">Third, the receipts. Again, if they had paid participants in both rooms, some would have had to have some of their payments reversed based on what happened in the second room, but none of the receipts indicate anyone gave any money back. And if they had paid participants in both rooms, there would have to be two receipts per person, or two sections of one receipt showing the first payment and later the second. The receipts have nothing like that either.<\/p>\n<p class=\"para\">Fourth, the structure of the data. If there were two payments, there would need to be two variables in the data set to record that. Maybe two rows for each participant, or one row with two numbers, two columns or with a comma. Well, there\u2019s nothing like that either. There was one row consistent with the obvious design that payment be made after the treatment once.<\/p>\n<p class=\"para\">Fifth, HBS second data expert acknowledged the weirdness in this charge, as he said, &#8220;the part that I find myself wondering about end quote,&#8221; on which &#8220;I think there is the most room for disagreement.&#8221; I can see why he was wondering, but I can\u2019t see why he thought there was any room for disagreement.<\/p>\n<p class=\"para\">So bottom line, we have a weird initial draft of the paper that described a brain-dead design for the experiment. But beyond that draft, there is no direct evidence to support the claim that this was, in fact, how the experiment was conducted, and there\u2019s clear indirect evidence that it was not done like that, both the logic and the structure of the data that was actually collected.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So then, what does Francesca say actually happened here?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Well, first, like any normal person, she has no actual recollection of what was going on with these changes 15 years ago. But second, she has said that if there were changes, they were changes to conform the paper to what actually happened in the room. Period.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so what\u2019s the exception? What evidence was there that the study was conducted as the original version of the paper described it?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">The Hearing Committee pointed to language used on an IRB form for this study.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, what\u2019s an IRB?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">That\u2019s the Institutional Review Board. It\u2019s basically the system to assure that research involving human subjects treats those human subjects ethically.<\/p>\n<p class=\"para\">Harvard pointed to language on that form that suggested the study was actually planned in the brain-dead way we discussed.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Is Harvard\u2019s claim that the researchers twice described the study as involving payments in both rooms, so that\u2019s two times the evidence suggesting that that\u2019s, in fact, what happened?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">That\u2019s their theory. But actually, it\u2019s clear that the language on the IRB form was just copy pasted into the write up of the results of the study, which then became an early draft of the paper.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">It\u2019s clear? That\u2019s exciting. Something in this case is clear. Why is it clear that it was just copy pasted?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">The study is described in bullet points in both documents, but there\u2019s an identical typo in both documents. They both say, &#8221; participants are welcome to the lab. Asked the read the consent form for the study and sign it.&#8221; That\u2019s the typo: asked the read rather than asked to read. That\u2019s why I say that it\u2019s clear that these are not two documents independently created, each describing payments in both rooms.<\/p>\n<p class=\"para\">This is one document that was created then copied into another, replicating the original typo in both places. There\u2019s nothing surprising about that: Researchers copy and paste across documents all the time. Many researchers feel some annoyance at having to write IRB proposals for studies that don\u2019t seem to present any kind of risk to participants, but by copying and pasting out of an IRB proposal, at least something useful comes from that process. And since the study is expected to be consistent between the proposal and what is actually done, ordinarily, this would be fine.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Did the description of payment in both rooms survive on the IRB submission?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">We don\u2019t have the submission, so we don\u2019t know what it said. The document that Harvard was pointing to is not from UNC IRB. It was simply a document found on Francesca\u2019s computer. That document was the form that would have been submitted to the IRB. We have no evidence whether, in fact, it ever was.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And so why don\u2019t we have that evidence?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Well, it wasn\u2019t in the evidence. Francesca reached out to the IRB o\ufb03ce and asked them if they had the final statements. The UNC IRB o\ufb03ce said they did not.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so basically, once again, we have an incomplete record about what actually was submitted to the IRB, and so no clear foundation to conclude, based on what was submitted to the IRB, that the experiment was as this original draft of the IRB form suggests. Certainly no reason to believe that this form was independently created, and therefore suggesting that there were two documents describing this brain-dead design.<\/p>\n<p class=\"para\">Instead, the text was copy pasted between the documents, and so we\u2019re left with still just one document describing a brain-dead design and no additional evidence to support the finding that, in fact, that\u2019s how the study was conducted.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Yeah, I would say that what was, or more accurately, what might have been submitted to the IRB doesn\u2019t change the way we look at whether the initial description actually described what actually happened in the room.<\/p>\n<p class=\"para\">Okay, so just to repeat and sum it up, because it really is quite extraordinary.<\/p>\n<p class=\"para\">Whatever was described originally, there is first no evidence from anybody who was actually in the room that there were payments done in both rooms. Second, no evidence in the forensic evidence, nothing in the receipts, nothing in the data files that suggest that payments were made in both rooms. And third, there is no godly reason why that\u2019s the way an experiment like this would have been designed, because that design would have no way to measure what the study was intended to measure. Causation flows in one direction. If a pledge of honesty is to have any e\ufb00ect, it\u2019s only going to have an e\ufb00ect in the future.<\/p>\n<p class=\"para\">So, when Harvard found the change in the description, it could either have believed (1) that the change was intended to conform the paper to what actually happened, or (2) that the change was meant to cover up what actually happened.<\/p>\n<p class=\"para\">But here\u2019s an example where there\u2019s not just no clear and convincing evidence of Francesca\u2019s guilt, but there\u2019s clear and convincing evidence of Francesca\u2019s innocence.<\/p>\n<p class=\"para\">You cannot, as a matter of law, conclude that the paper was covering up anything when, (1) there was no evidence beyond the initial draft of anything to be covered up, and (2) that the thing that would be covered up makes no freaking sense as the design for an experiment.<\/p>\n<p class=\"para\">As between the two possibilities, (1) that the change was meant to conform the paper to the experiment, or (2) that the change was meant to hide a brain-dead design for the experiment, only the most biased Fact Finder could conclude number two over number one. And certainly, no fair Fact Finder could conclude with clear and convincing evidence that these talented researchers, later joined by two other researchers, would design such a stupid study and then attempt to cover up their own design stupidity.<\/p>\n<p class=\"para\">To the contrary, there was a typo in an earlier draft of a paper describing the research project. That typo was corrected.<\/p>\n<p class=\"para\">It should be chilling to any academic that the evolution of a draft paper 15 years ago would be foundation for the prosecution to remove someone\u2019s tenure.<\/p>\n<p class=\"para\">Okay, that\u2019s allegation number four.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Ava, thanks so much for participating in this conversation. I look forward to talking more about the remaining two allegations.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Thanks.<\/p>\n<p class=\"para\">That was episode five of season four of the podcast, &#8220;The Law Such As It Is.&#8221; We will consider in the next two episodes, two more allegations against Francesca, and then there will be at least one final episode, pulling it all together and suggesting, where do we go from here.<\/p>\n<p class=\"para\">You can find these podcasts wherever you find podcasts. There\u2019s a website, theginocase.info, that has source material behind the podcast. There\u2019s also a sub stack you can find if you sort search for \u2018the Gino case\u2019 with my name attached to it.<\/p>\n<p class=\"para\">I hope you will follow and share with people who might be interested to follow these facts. I\u2019m grateful to my friends that they would help me unpack this story. I\u2019m even more grateful that they will help me unpack the allegations in the next two episodes.<\/p>\n<p class=\"para\">Thanks again for listening.<\/p>\n<p class=\"para\">Stay tuned, I hope not too long from now for the next episode.<\/p>\n<p class=\"para\">This is Larry Lessig.<\/p>\n<\/div><footer class=\"ep-footer\"><a class=\"ep-footer-link\" href=\"#gino-index\">\u2191 Back to index<\/a><a class=\"ep-footer-link next\" href=\"#gino-episode-7\">Next: Allegation #3 \u2192<\/a><\/footer><\/section>\n<section class=\"episode\" id=\"gino-episode-7\" data-screen-label=\"07 Allegation #3\"><header class=\"ep-header\"><div class=\"ep-eyebrow\">Episode 07 &middot; The Law, Such As It Is &middot; Season 3<\/div><h2 class=\"ep-title\">Allegation #3<\/h2><p class=\"ep-blurb\">Examining the evidence behind allegation number three.<\/p><\/header><div class=\"ep-body\"><div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para has-dropcap\">This is Larry Lessig. This is the seventh episode of season three of the podcast, &#8220;The Law Such As It Is.&#8221; In this season, we are reviewing the decision to revoke the tenure of Francesca Gino at the Harvard Business School.<\/p>\n<p class=\"para\">That decision was based on four claimed acts of academic misconduct, basically data fraud. And so far, we have addressed two of those four charges. In this episode, we\u2019re going to address allegation number three, which will then leave just one final allegation to review, allegation number one.<\/p>\n<p class=\"para\">If you\u2019ve been listening to this podcast, the critique of allegation number three is going to repeat some of the same themes that we\u2019ve covered in allegation two and allegation number four.<\/p>\n<p class=\"para\">First, as with allegations 2 and 4, the claim of the Hearing Committee with this allegation is that the anomalies identified in the data all tended to support the conclusions of the paper. That claim by the Hearing Committee in this case is once again and overwhelmingly false.<\/p>\n<p class=\"para\">Second, as in the earlier allegations, the Hearing Committee concluded that Francesca must have been responsible for the changes identified. Once again, despite it being uncontested, the data cleaning, coding and processing was work done by research assistants \u2014 and in this case, research assistants working with the data for more than 500 days. Never once were those research assistants questioned about their work with the data.<\/p>\n<p class=\"para\">That\u2019s not all that\u2019s wrong with this allegation, but it is a start. Let\u2019s get to the particulars.<\/p>\n<p class=\"para\">Once again, I\u2019m joined in this conversation by an anonymized version of a friend who has studied this allegation carefully, as I described in the last episode. My friend\u2019s voice has been modified by fancy AI, so this is not what my friend sounds like in real life, although I gotta say the AI is not too bad either. So welcome Ava.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Thank you, Professor.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Why don\u2019t you introduce the allegation?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Sure. Allegation three involved a paper titled &#8220;Evil Genius: How Dishonesty Can Lead to Greater Creativity.&#8221; The paper was published in Psychological Science in 2014, and the aim of the paper was to examine whether there was a correlation between acting dishonestly and creativity.<\/p>\n<p class=\"para\">In one study in that paper \u2014 study number 4 \u2014 participants played a game where they were to guess the outcome of a virtual coin flip. After they guessed the outcome, they flipped the virtual coin, and then they were to report whether they had guessed it correctly or not based on the first flip. They were asked to flip the virtual coin a couple more times to ensure the toss was legitimate. The software knew what the outcome of the virtual flip was, and so the researchers could tell whether the participant reported it correctly or not, meaning the researchers could tell whether the participant lied or not. The participants then completed some creativity tasks. The hypothesis was that those who cheated would be more creative.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And is that what they found?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Yes, that\u2019s what the study found. Participants who cheated performed higher on measures of creativity.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So when was the data for this study collected?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Though the paper was published in 2014, the study in question, study 4, was conducted in late 2012. So we\u2019re back to some of the issues you covered in prior episodes when you talked about the reasons behind the six-year statute of limitations on reporting research misconduct: it is really hard to reconstruct exactly what happened almost 14 years after the data was collected. And as we\u2019ll see, that point applies to this allegation especially.<\/p>\n<p class=\"para\">Right, I\u2019ve said plenty about the limitations point: The University promises that academic charges of academic misconduct six years or older &#8220;will not be investigated.&#8221; But literally millions have been spent investigating these charges, some by Francesca and multiples of that by the University. But we\u2019re going to pass over that flaw for now.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Let\u2019s start with the context of when this paper was written, because this is a paper where there\u2019s particularly strong evidence that there were certain errors made before publication. Is that right?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Yes. They are what we\u2019ve been talking about throughout this podcast: the anomalies. It\u2019s clear there are anomalies in this paper. The question is what the source of those anomalies is.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">You mean the question is whether Francesca herself introduced those anomalies, or whether they were introduced by the research assistants. Correct?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Correct.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And why would you think that the anomalies would have been introduced by research assistants?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Well, remember that one constant theme in this podcast has been the Hearing Committee\u2019s constant mistake in claiming that the anomalies they\u2019ve identified all tended to strengthen the conclusions of the paper. That\u2019s not true. It\u2019s not true in the other allegations we\u2019ve considered. It\u2019s not true in this case either.<\/p>\n<p class=\"para\">If you have anomalies that don\u2019t strengthen the conclusion of the paper, that would lead you to think that these are not anomalies that Francesca herself would have intentionally introduced into the paper. You typically don\u2019t commit academic fraud by intentionally reducing the significance of the results you\u2019re discussing in the paper. Instead, anomalies that weaken the conclusions of the paper would more likely have been made by research assistants.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">What did the research assistants say about this allegation?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Nothing.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">What do you mean, nothing?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Well, the research assistants said nothing because the research assistants were not interviewed with this allegation.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">The research assistants were not interviewed?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">No. The research for this paper continued over two years, there were at least five separate research assistants who worked on the research, but Harvard did not interview any of them.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So let me make sure I understand this correctly. We know that the anomalies could have been introduced either by Francesca intentionally or by research assistants accidentally. We also know that the anomalies in this case are not all supportive of the conclusions of the paper, so some of the anomalies here are most likely the result of the research assistants. To understand whether and how the research assistants could have introduced these anomalies, a proper investigation, a complete investigation, would have had to consider the work of the research assistants, but Harvard did not do that. Instead, Harvard raced to the conclusion of guilt at the business school stage and then tripled down with 100,000 lawyers to affirm that conclusion throughout this multi-year process.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">100,000 lawyers sounds like a lot, but yes, it\u2019s a good summary and an important summary, because the point should be obvious: no good prosecutor\u2019s office would think it had completed its investigation when it hadn\u2019t even spoken to people who could have easily created the problem being investigated.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">I don\u2019t think anybody has ever accused Harvard of being a good prosecutor\u2019s office.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Fair enough. But I want to make an even stronger point: where the evidence exists in this allegation, we can say with certainty that it was the research assistant who introduced the anomaly.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And how can you say that?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Let\u2019s divide the research into two parts \u2014 an objective part and a subjective part.<\/p>\n<p class=\"para\">The objective part of this study, study 4, related to a virtual coin flip. Participants were asked to predict the results of the coin flip. They indicated their prediction. The coin was flipped. They were asked whether they had guessed it correctly or not. Most people told the truth. Some people lied \u2014 some said they had guessed it correctly when they hadn\u2019t. The aim of the study was to see whether those who lied were more creative than those who didn\u2019t.<\/p>\n<p class=\"para\">The subjective part that\u2019s relevant here can itself be further divided into two parts, each involving a creativity task participants completed and that research assistants then needed to code.<\/p>\n<p class=\"para\">The first task involved a remote association test with a tight time constraint of five minutes. Participants were given three words and asked to provide a word that was suggested by the three words.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So how would that work?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Well, for example, the participants would be given &#8220;blank-white-lines&#8221; and then asked to enter a word that connected all three \u2014 in this case, &#8220;paper&#8221; is a logical choice.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And so how many questions like that were there?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">There were 17.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So, a participant would give a word association for 17 triads of words.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">That\u2019s right.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And then how would they be evaluated?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Well, first, the RA would have to clean up the data. Sometimes the participant would misspell the word p &#8211; a &#8211; p &#8211; a &#8211; r rather than the right spelling of p &#8211; a &#8211; p &#8211; e &#8211; r, for example. Or sometimes they\u2019d put a question mark at the end of an answer they were unsure of. Regardless, the RA would go through the Excel sheet and clean up the data to make it consistent and meaningful.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And when was this cleaning done?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">We don\u2019t have the original Qualtrics data, but we do have a file that was created about 11 hours after the study was completed. That file was saved on November 18, 2012; the file used for analysis before the paper was submitted for the first time is from July 24, 2013 \u2014 almost 250 days later.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, but you said this was the first bit of the subjective part of the research. Was there a second bit, or a second part to this subjective part of the research?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">There was. The first subjective part we\u2019ll call the RAT task, for &#8220;remote association test.&#8221; The second, we can call it a &#8220;Usage Task.&#8221;<\/p>\n<p class=\"para\">In addition to being asked to make a word association with three given words, the participants were also asked to name as many uses of a newspaper as they could in one minute. Like the RAT, this is a common creativity task, often used in the psychological literature.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">The uses of a newspaper?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Yeah. Like, what could you do with a newspaper? I mean, the obvious use would be to read it, but you could make a paper airplane with it, or use it to start a fire, or frame it, or wrap a fish in it. I guess, lots of things you could do with a newspaper.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And so why were the participants asked to supply possible uses of a newspaper?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">The goal was then to grade the creativity of their replies. Not only the number of distinct uses people came up with, but also how different they were from one another, and how original they were. If you said you would read the newspaper, that\u2019s a solid, boring, uncreative use of the paper. If you said you could use it to train puppies or make a paper hat, those would be very creative uses of the paper. The objective was to distinguish between people who could identify many creative uses of a newspaper and those who could just identify a few.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Right, because the question of the study was whether liars were more creative.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">That\u2019s right. I mean, we know they\u2019re creative with the truth. The question was whether that creativity carried over beyond the truth.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, but this grading of these answers feels a little bit subjective, doesn\u2019t it?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">It\u2019s certainly subjective, and that created an issue in the preparation of the paper. Initially, when the authors submitted the paper to the journal that eventually published it, they received what\u2019s called a &#8220;revise and resubmit&#8221; response. That means the journal liked it, but they wanted it revised, and the journal described what that revision should be.<\/p>\n<p class=\"para\">As it related to these creativity measures, the journal suggested that they evaluate the creativity along two additional dimensions of creativity. So rather than just one measure of creativity, there would be three. That\u2019s where the subjectivity issue arose: If the responses for the first dimension were evaluated by one person, but then the responses the next two dimensions were evaluated by a different person, the difference between those evaluations could produce bias in the results. To avoid that bias, after this revise and resubmit request, the authors asked for one RA to evaluate all three dimensions. That RA thus produced a new evaluation along all three dimensions.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, but then what was the committee\u2019s allegation of fraud here?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Here\u2019s where the difference is between these two textbased responses \u2014 the RAT task and the Usage task \u2014 are so illustrative. Because we actually have the original coding data for the Usage Task. And so we can actually see where the error for this task arose.<\/p>\n<p class=\"para\">If we compare how the research assistants coded the data \u2014 after the revise and resubmit, the three dimensions of data \u2014 to the final data set that was submitted, we can see a number of places where the final data set was produced by copying the wrong part of the original data set.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">What do you mean by that?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">I mean that the person who imported or copied the data into the final dataset \u2014 and there were two, because this part of the data was imported at two different points of time, one initially, and one after the revise and resubmit request \u2014 that person copied data from the wrong part of the source dataset into the final data file. With the first error \u2014 what the hearing committee identifies as 7 values that had been changed \u2014 we can show the copying is from the right tab of the data file, but the wrong rows. With the second \u2014 what the hearing committee identifies as 18 other errors \u2014 we can show that the error was in the copying of the data from the wrong tab of the source data sheet.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Right, I read the that the committee said the data was copied from a different study into this study, and that they used that to evidence the fraud.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">With respect to the 18 participants, yes, it was copied from the wrong tab or wrong rows.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And so you\u2019re saying that was error. And again, how do you know that it\u2019s just error?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Because the consequence of that copying was to weaken the study, not to strengthen it.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">It weakened it. The anomaly weakened the study, right?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">It weakened it. So the idea that this copying was made as an act of intentional falsification of fraud is just wildly implausible. There was no motive to falsify data to weaken findings.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so that\u2019s with respect to the 18 participants in the usage test. What about the other seven here?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">The difference did make the result slightly stronger. But even here, if you reverse the mistake\u2013 if you put the original text back with the correct participants \u2014 the result is still highly significant. The change thus did strengthen the results, but they strengthened an already significant result. Which means there could have been motive, but not much: there was no need to strengthen the paper at that point, and in that way, because the paper was already strong enough.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so with the usage tasks, your claim is that you can point to exactly where in the source file the data was copied from, and because of that, we can make an inference about whether the copying was fraud or not?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Exactly. With respect to the largest error, the one affecting 18 participants, the copying looks like an error. We can find the place where the data was copied from. The copying weakens the strength of the creativity measure and the overall strength of the study. Why would you falsify data to weaken the conclusion of a paper? You wouldn\u2019t: which is why I\u2019m saying this is likely an error.<\/p>\n<p class=\"para\">With respect to the seven, we can see the values were taken from the wrong row of the correct study. Here, there\u2019s an increase in the statistical significance of the study, p equals .03, down to p less than .01. So that suggests a possible motive.<\/p>\n<p class=\"para\">But arguing against that possibility is a simple fact: the finding gets would be strong either way. There is a difference, but not a difference that either reviewers or readers would ever worry about or even think about.<\/p>\n<p class=\"para\">Thus, for the first change, there was no motive for the change. And for the second, there was, at most, an awfully weak motive. And from my perspective, the presence of the first undermines any negative inference about the second.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">What do you mean by that?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Well, again, if the anomaly weakens the finding, that suggests errors. Not all errors are going to be in one direction, so it\u2019s not surprising that the second error was in a different direction, which means the first error weakens the suggestion of fraud in the second.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">With respect to this set of anomalies.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Right. We\u2019re just talking about one of the two sets of subjective evaluations. One of those two is most likely error, not fraud.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And what about the other subjective evaluation?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Well, here, unfortunately, we don\u2019t have the same source data.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">What do you mean?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">I mean, with the other subjective task, we had the coding data from the RAs who coded it. Here, we don\u2019t. So here, we can\u2019t be as sure about what exactly explains this error.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, well, so first describe this error.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">In this part of the charge, there are four participants out of the 178 whose scores basically don\u2019t add up.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">What do you mean by the scores don\u2019t add up?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Remember, this is the RAT task, the task where participants were given three words and were to answer what those three words were associated with. The example I gave before was &#8220;blank-white-lines&#8221;; and for those three words, the clearest answer was &#8220;paper.&#8221; For these four participants\u2014 out of 178 \u2014 if you look at the individual coding from Excel for each of the 17 answers to see whether the participants got the right answer, they don\u2019t add up to the aggregate or the total represented for all the answers together.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Why would that be?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">We can\u2019t figure this part out. Part of the problem was caused by a mistake in the coding of the original file. Remember the original file \u2014 the file produced 11 hours after the study was over \u2014 is the closest we have to the raw data. But that file already has coding in it.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">It already has coding in it?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Yes, it isn\u2019t just the raw data from Qualtrics. It included coding \u2014 meaning formulas designed to convert the answers into usable data. That means that file had already been worked on by RAs. And that means, as with allegation 4, we don\u2019t actually have the original data, but are making inferences from a file that had already been worked on.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so what were the mistakes in that coding?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Well, we can say that the formula that tested the response was wrong in at least two cases. For example, one answer was supposed to be &#8220;thief,&#8221; but the coding formula spelled thief incorrectly. E, I, rather than I, E. Another answer was supposed to be &#8220;beer,&#8221; but the coding formula spelled &#8220;beer&#8221; as &#8220;bear,&#8221; B, E, A, R. But even after you correct these formulas, you don\u2019t fix the problem. With respect to these four participants, the totals for these four still don\u2019t add up, and because we don\u2019t have the RAs\u2019 coding sheets, we can\u2019t say why these four don\u2019t add up correctly.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Did the changes strengthen the paper?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Yes, they did. These changes did marginally strengthen the paper. P was .012, versus p equals .04. But here again, if you took these four participants out of the study, both with these subjective questions as well as the uses tasks, it wouldn\u2019t have changed the finding of the paper. The paper was statistically strong without them, and both could legitimately be reported as p less than .05, so again, the referee and the readers would not know. This again raises the question: why commit fraud when it doesn\u2019t change the publishability of the ultimate paper?<\/p>\n<p class=\"para\">Okay, but here again, they didn\u2019t talk to the research assistants. Conceivably, the research assistants would have had the coding sheets that would have made it possible to see exactly how the error was made, if indeed it was an error. If they talked to the research assistants, there was a chance those research assistants would still have had the coding data if they had the coding data. Then, as with the first subjective scoring, you could have figured out why. In this context, there was also an error. Harvard didn\u2019t interview the research assistants and didn\u2019t think it was important to have a complete investigation. So we don\u2019t have that information here.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so summing up the story about these subjective tests: With respect to the first, because we have the coding data, we can be pretty confident that this was just an error by the research assistants. With respect to the second, we don\u2019t have the coding data, so we can\u2019t absolutely negate the idea that this was intentionally, fraudulently tweaked. But even here, if you remove the observations for the four scores that don\u2019t add up, the paper would have been statistically strong even without them. So there\u2019s a motive, but not a clear motive.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">That\u2019s right.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so let\u2019s turn to the objective test. Explain again the issue around this virtual coin flip.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">In this part of the study, participants were asked to guess what the results of a coin flip would be. They then flipped the coin and could see the result. They were asked next whether they had predicted correctly how the coin flip would come out. The majority of them told the truth, and about 24% of them lied.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And I take it the study knows this because the program recorded what they had guessed, what the result of the coin flip was, and whether they said they had guessed correctly.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Maybe. You see, we just don\u2019t know exactly what the program did, because we don\u2019t have the program, and Harvard never asked the programmer who wrote it to provide the program.<\/p>\n<p class=\"para\">But the more I\u2019ve studied this allegation, the more I\u2019ve realized that both sides just misunderstood what was happening here. Both sides were really talking past each other, and we can simplify the story pretty dramatically if we assume that what happened is what Harvard said happened. Again, we don\u2019t know that\u2019s what happened. We don\u2019t have the data because we don\u2019t have the program. But the point is, even if we accept Harvard\u2019s description of what happened, we can show this is not fraud.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so what did Harvard say happened here?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Harvard said the coin flip program was rigged.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">How could it have been rigged? What does that mean?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">It\u2019d be rigged to basically give the opposite of whatever you guessed. If you guessed heads, the program would simulate a flip and show you tails. You guessed tails, the program would simulate a flip and show you heads. That way everybody in the survey thought they had gotten it wrong. The question then was whether they would tell the truth about whether they got it wrong and how their truth telling would affect their creativity.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so then what was the problem here?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Well, there are 178 participants in this data set. On Harvard\u2019s assumption about how the study was conducted, all of them were told they got the coin flip wrong. When they were asked whether they had gotten it right or wrong, the majority were honest and reported that they hadn\u2019t guessed it correctly. But about 24% lied. About 24% said they had guessed it correctly, when in fact they hadn\u2019t, because, again, on Harvard\u2019s assumption, the coin flip was rigged, so everyone was told they had gotten it wrong.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, and so what\u2019s the problem with that? Where was the source of the alleged manipulation of data here?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Well, there were 12 participants who, like the majority, had said they had not said that they had guessed it correctly. They had been honest about the fact that they hadn\u2019t guessed it correctly. Nonetheless, these 12 were marked as cheaters. Or more specifically, between the first file that we have available and the final file we have available. These 12 went from being marked as non-cheaters to being marked as cheaters. Harvard\u2019s claim is that the change was itself cheating, allegedly by Francesca.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">What do you mean marked as cheaters?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Well, remember the file we have here is a file that was produced about 11 hours after the study was completed. And as I said, it wasn\u2019t the raw data from the Qualtrics survey, because the file already has coding in it.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Remind me again, what do you mean by coding in it?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Well, already there are formulas that the RA had put into the file to begin to process the data, to make it usable to conduct the study. Formulas that would translate the raw responses the participant had given into usable data that could be analyzed.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so what data was there related to cheating?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">In this file, there was one variable or column in the data set called &#8220;reported guessed correctly&#8221; that reported whether the participants said they had guessed correctly or not. If they didn\u2019t say they had guessed correctly, then the value of that variable was zero. If they said they had guessed correctly, the value of that variable was one. The people who said they guessed correctly were lying. So those people, at least we know, were cheaters. These 12 participants, like the majority, had honestly reported that they had not guessed correctly. So these 12 had a zero in their &#8220;reported guessed correctly&#8221; column in the first version of the file.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So again, no one had guessed correctly, because on Harvard\u2019s assumption about how this data was created, the survey was rigged. If you had a zero in the &#8220;reported guess correctly&#8221; column, you\u2019re telling the truth, and if you had a one, you&#8217;re telling a lie. And presumably, the study treated those who had a one in the &#8220;reported guess correctly&#8221; column as liars or cheaters. Is that right?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Almost. It turns out, right next to the &#8220;reported guessed correctly&#8221; column, there was a second column. That column was marked &#8220;cheated.&#8221; In the first version of the file produced 11 hours after the study was complete, the values in that column were the product of a simple formula.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">A formula?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Yes, a formula. A formula to calculate whether that person would be marked as cheating.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And so what was that formula?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">The formula looked at the value in the previous column, the &#8220;reported guessed correctly&#8221; column, and, if the value was zero, meaning they didn\u2019t say they had guessed correctly, then the value in the cheating column was zero, meaning they hadn\u2019t cheated. If they said they had guessed correctly, then the value in the cheating column was one.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So you\u2019re saying the values in the &#8220;cheated&#8221; column were exactly as the same as the values in the &#8220;reported guessed correctly&#8221; column?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Yes, there was a formula that just carried over the values from one column to the other.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so that\u2019s weird. Why would there be two columns reporting essentially the same data?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Well, this is the point that no one seemed to ever try to work out. Why indeed would there be two columns reporting the very same data? But as I\u2019ve studied this more, it seems pretty clear why there were two columns. That\u2019s because whether someone was a cheater did not depend solely on whether they said they had guessed correctly.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Well, what else could it have depended on?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Well, certainly it depended primarily on whether they had lied on having guessed the coin flip correctly. But the fact that there was a second column shows pretty clearly that there could be some other reasons why someone could be marked as a cheater.<\/p>\n<p class=\"para\">For example, the survey itself had rules. If you could determine that the participant was violating those rules while completing the survey, then yes, that participant was a cheater, not necessarily because they had lied about the coin flip, maybe they told the truth there, but because they had violated the rules of the survey.<\/p>\n<p class=\"para\">And the point is that if you wanted a way to mark more people as cheaters than simply those who reported that they had guessed correctly when they hadn\u2019t guessed correctly, you would need two columns to do that. The first reporting the lying about the coin flip, but the second, giving the RA freedom to mark more than the coin flip liars as cheaters.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so is there any indication that the 12 who had been marked as cheaters were cheaters for some reason other than that they lied about the coin flip?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">There sure is. There are indications in the data to suggest that these 12 might have been cheaters, even though they reported truthfully that they did not guess correctly.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">How else could they have been properly marked as a cheater, if, in fact, they reported honestly that they did not guess the coin flipped correctly?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Remember the people answering these questions are being recruited from something Amazon runs called Mechanical Turk.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">What\u2019s Mechanical Turk?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Mechanical Turk is a site, really a complicated online application where you can earn money by participating in studies that people are running, psychology researchers, among many others. You sign up, you get tasks to do. You do these tasks, and you get paid based on the number of tasks you complete. Some people do it for a hobby. Some people do it for extra money. I imagine some people in some countries are living off the money.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, but what does the fact that people are coming from Mechanical Turk have to do with someone\u2019s incentives to cheat or not?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">If you\u2019re a Mechanical Turk worker, you want to be hired. You\u2019re not going to be hired if you\u2019re seen as a flake. In theory, you could go through a task and give the same answer to every question, yes, yes, yes, yes, but the people evaluating your answers will mark you as not a serious worker, that will make it harder for you to get hired again. Specifically for questions where there is an objective answer, you need to get that answer right, or at least close to right, otherwise, your reputation as a reliable MTurk worker would go down.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So for questions with an objective answer, how do you make sure to get the answer right?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Ask any teenager doing their homework on a computer connected to the internet, you go to Google, you search for the answer.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">How are you going to search up the answer to a survey being run by a business school professor about word associations?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Well, it turns out that this particular word association task, what we\u2019ve been calling the RAT task, is common in psychology. So there are lots of examples of it on the web. A Mechanical Turker eager to get the right answers could just search these answers and copy the answers from the web into the submission.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, but that would suggest that these 12 did really well with respect to those objective tests. Did they do well?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Yes, they did. Their overall mean performance score (the RAT_perf variable in the data) was 11.2 compared to 7.7 for the rest of the participants. Running a statistical test, a t-test, confirms this difference is highly significant, P equals .02. And 100% of these weird 12 successfully solved specific difficult puzzles like RAT2 and RAT3, that stumped over 30% of the broader group. That\u2019s telling, and it suggests they might have used external help, like a web search. That meant they were doing something against what they were told for the RAT task, where the instruction said, and this is a quote, &#8220;Please do not use any help other than your own knowledge.&#8221;<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So that is saying that these 12 were particularly good with respect to these RAT tasks. But why does that show that they were cheating?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Well, they weren\u2019t particularly good with the other creative task, the one that\u2019s harder to look up on the web. On the usage tasks, these 12 were just normal. The 12 generated an average of 6.83 alternative uses, while the rest of the group generated an average of 6.61 alternative uses. A statistical ttest shows this difference is insignificant, P of 0.77.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So you\u2019re saying that an RA could notice this difference and reason that these 12 were cheaters because they were not following the instructions they were given about how they should answer the questions. Namely, not looking the answers up, because they were obviously looking the answers up when they were told not to. It was reasonable to call them cheaters.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Exactly. If the cheat variable measured more than just whether they lied about the coin flip, then an RA could look at this and believe that it identified participants who were cheating.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">But wouldn\u2019t it take more time to go to the web and get answers than just to give a first guess of whatever the answer should be?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Yes, it would.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And did these workers actually take longer to complete their survey than the others did?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">They sure did. These 12 took longer, on average, to complete the survey (about 21 minutes) compared to the rest of the group (about 15 minutes). If they were simply click farming bots, they\u2019d be faster. A few minutes ago, I posited maybe they were inattentive, clicking Yes, yes, yes, yes, yes, almost randomly, in a hurry to get the work done. But if inattentive users would maybe do the work in a hurry. Well, these 12 actually took longer. And if these 12 were humans, reading a puzzle, opening a new browser tab, typing the words into a search engine, reading the results and typing it back into the survey, their completion time would naturally increase.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Wait, wasn\u2019t there a five-minute time limit for the RAT? And if there were a time limit, and most subjects used all the time for this tricky task, then how could these 12 take longer to complete the task?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">It turns out Francesca\u2019s so-called time limit was on the honor system. That created an additional opportunity for participants to cheat \u2013 take more than the budgeted time on the survey page. With the RAT test, Qualtrics automatically recorded the timing of the very last click on that page before the final submission and the timing of the very first click on the page. Looking at the difference between the two times, you see that the 12 participants took 436 seconds on average, that\u2019s a bit over 7 minutes, while the rest took 340 seconds on average, well under 6 minutes. A t-test confirms the difference is statistically significant, P equals .07, though only marginally different.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So the theory is that these research assistants noticed that the higher RAT scores, including the perfect scores on two of the hardest RAT questions, plus the fact that it took these 12 subjects longer, on average, to get the answers and the fact that they were not great on the other usage tasks. Together, these indicated that they were possibly or likely cheaters. Are there other indicators that these are the sorts of people who are likely to be cheaters?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Yes, at least one. These 12 scored low on a test designed to measure caring about the rules. Questions like, &#8220;if a supervisor or person with authority gave me some direction, right now, I would follow them.&#8221; So that would be consistent with them being found to be cheaters, given the other evidence that they had looked up the answers for at least some of the questions.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so you\u2019re saying adding all these things together, an RA could have looked at these different signals from the results these participants gave, and determined that these participants were cheaters, even though they did not say that they had gotten the coin flip right.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">That\u2019s right. And this way of understanding it makes sense of the second column. When you ask, &#8220;Why is there a second column marked \u2018cheated\u2019?,&#8221; the answer must be that &#8220;cheated&#8221; was not simply a function of whether they said they flipped the coin correctly or not. Instead, that &#8220;cheated&#8221; column would have been determined based on a number of factors, maybe first by whether they lied about getting the right coin guess on the coin flip, but not just that. The fact there is a second column means the study required the RA to make a broader judgment.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, but are there other counter examples? Are there other participants who fit this particular pattern, who maybe were not marked as cheaters?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Yes, I don\u2019t want to overstate the claim. I\u2019m not claiming that this is a clear rule that the RA applied 100% consistently. All I\u2019m saying is this: if the RA had been asked to make a judgment, these 12 would have been appropriate for the RA to mark as cheaters. Again, we don\u2019t know whether the RA was asked to make a judgment, because, again, Harvard did not interview any RA for the study. The critical exculpatory evidence was just never collected.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So this is a great example of why even a very simple interview would have gone a long way in resolving what actually happened here. While an RA is not going to remember every detail of working through a dataset, they might well have remembered whether they had been asked to make a judgment about whether a participant had cheated, beyond simply recording whether they had reported correctly that they guessed the coin flipped correctly. They might have remembered concluding that someone had used Google to figure out the RAT task. This is an obvious example about how failing to do the most basic investigation wildly undercut the fact finder\u2019s ability to know what exactly happened.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Right, if they had interviewed the research assistant and said to the research assistant, &#8220;What led you to mark someone as a cheater when they hadn\u2019t falsely claimed to get the coin flip toss right?&#8221;, that research assistant could either have said, &#8220;I didn\u2019t make any changes in the cheating field column,&#8221; which would tend to incriminate Francesca. But if the research assistant instead said something like &#8220;I was applying a more multi-dimensional analysis, I was looking at this other stuff too,&#8221; well, that would tend to exonerate Francesca. Imagine if the research assistant gave remarks like what we described above, how RAs all know that RATs can be gamed, so if you see scores too good, plus a long time, you can infer cheating. But once again, a basic part of the investigation was just not conducted because Harvard assumed that she was guilty. They weren\u2019t investigating whether she was guilty. So yes, they didn\u2019t even interview the RAs.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Wow. Okay, so then what about the third part of the charge made by the Hearing Committee?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Here, the committee found that Francesca had fiddled with the data to hide her earlier fraud, or what they said was her earlier fraud when one of the researchers from Data Colada asked her to provide the underlying data for this study.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And did she alter the data?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Here again, as with the rest of the charge, it\u2019s impossible to know with confidence without talking to the research assistants involved. Because Francesca is clear about one important fact: she said she did not create the data file that was shared with Data Colada.<\/p>\n<p class=\"para\">The question for the research assistant would then have been quite simple, &#8220;did you create the data file or not?&#8221; If the research assistant said, &#8220;Yes, I worked on the data file,&#8221; then it would be important to understand what the research assistant had done and why. But if the research assistant had said, &#8220;No, I didn\u2019t work on the data file, I remember Francesca said she would take care of it,&#8221; that, again, would have been completely damning for Francesca. The failure to do this most basic step of investigation means we don\u2019t have an answer to this fundamental question.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, but why did the Hearing Committee conclude that her manipulating this data was fraud?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">The Hearing Committee flagged two changes that happened in the data. There were others. For example, column headings were clarified to make it easier to understand the data. That is the sort of work that research assistants would do when preparing to post data publicly or share it. But beyond that innocent change, there were two other changes that the committee didn\u2019t find to be innocent.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And what were the non-innocent changes?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">The basic claims that the data provided to Data Colada had been modified in ways that clumsily covered up earlier modifications. Either covered up or corrected, depending on the scenario.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And did Francesca acknowledge that these changes had been made?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Again, what Francesca insisted upon was that she was not in the business of preparing data to be shared with anyone. When a request for data was made, she would have passed it to an RA, and the RA would have prepared the response, which she would then forward along. But no one denies that these anomalies exist. The only question is, who is responsible for the anomalies and why?<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, that could have been her ordinary practice. But in this case, if we assume that the earlier modifications were examples of fraud, we can see why she would have had an incentive to tweak these data to make sure that the fraud was not revealed. Isn\u2019t that right?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">That\u2019s right. If you don\u2019t believe Francesca, then you believe that what happened is she got this request for the data, panicked, realized that she needed to cover up the earlier alleged fraud, and then tweak the data to cover up that earlier misconduct. But then you must also believe that her cover up was clumsy because she was caught because of the inconsistencies that the cover up revealed.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so then help us understand what exactly were the changes that either she or the research assistant had made.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">There were two things that were changed at this stage. The first related to the 12 participants who had before been marked as cheaters, even though they had not asserted that they had guessed correctly. For these 12 participants, the data given to Data Colada modified the &#8220;reported guessed correctly&#8221; variable to set it to a number that was the same as the cheating variable.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, just to be clear here, to remember what we were talking about: the &#8220;reported guessed correctly&#8221; variable is zero if they didn\u2019t say that they had guessed it correctly, and one if they said that they had guessed it correctly. That data came from Qualtrics right, that came directly from the survey. The &#8220;cheated&#8221; variable was the derived variable. Derived in part, we suggested, from the &#8220;reported guessed correctly&#8221; variable (if you reported guessed correctly, then you are marked as a cheater), and in part, as we\u2019ve concluded, from other factors.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">That\u2019s right.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And what you\u2019re saying is that the modification that was made to the data before it was given to Data Colada was to change the Qualtrics data to conform to the result in the &#8220;cheated&#8221; variable column. Is that right?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Right. The changes to a column that comes originally from Qualtrics, someone changed that column, the values in those rows in that column, to make those entries consistent with the &#8220;cheated&#8221; variable. If that someone was Francesca, it would look like an effort to cover up earlier data manipulation. If it was an RA, it could simply have been an RA who didn\u2019t understand what the &#8220;cheated&#8221; variable was, who was trying to understand why the field seemed to be inconsistent. That RA definitely should have flagged the problem to Francesca, but didn\u2019t. Instead, the RA just tried to make the data consistent before sharing it with Data Colada.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Is there a reason in this strategy to believe that it is or isn\u2019t a strategy that a fraudster would have deployed?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Well, if this was a cover up, it would have been a cover up by basically the dumbest means possible. If you were responding to a request from a fraud investigator in a way designed intended to hide your fraud, you could simply have deleted the &#8220;reported guess correctly&#8221; column, since it wasn\u2019t used in the analysis.<\/p>\n<p class=\"para\">Or you could have changed the &#8220;cheated&#8221; variable back to match the reported guess correctly column, making it look like the reclassification never happened.<\/p>\n<p class=\"para\">Those two strategies would have eliminated the discrepancy, but preserved the original data, which of course, the fraud investigator would have had access to.<\/p>\n<p class=\"para\">Instead, what the modifier did was the opposite: they left the &#8220;cheated&#8221; column alone, the &#8220;cheated&#8221; column that had been changed, at least on this theory, and change the raw selfreport data, the part that comes from Qualtrics. This just preserves the fabrication and destroys the original data, giving the data fraud investigator the evidence they need to claim something fishy is going on.<\/p>\n<p class=\"para\">So yeah, that sounds pretty stupid. And if you conclude this was indeed fraud by Francesca, it shows Francesca is not only fraudulent but stupid in her fraud.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">What about the changes related to the four creativity scores?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Remember here, the charge was that the numbers didn\u2019t add up for these four participants. The total RAT score is not the sum of each RAT1 through RAT17. Here too, the charge is that someone changed the individual numbers so that they did add up.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And if that someone was an RA, why would an RA have done that?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">If it was an RA who had been asked to pull together these data, and if he or she saw these inconsistencies, then they plainly should not have done what they did. They should have flagged the inconsistency and asked Francesca about it. But if they didn\u2019t, then these changes were clearly aiming to make the data consistent. If the RA didn\u2019t know much about what I explained regarding the inconsistencies, then they were just cleaning up the data to make it consistent. It would have been an error, a pretty serious error, but it would not have been fraud by Francesca.<\/p>\n<p class=\"para\">But if, on the other hand, it had been Francesca, then the changes were being made to hide the suggested fraud on the four aggregate scores that had been raised.<\/p>\n<p class=\"para\">And you know, for both sets of changes, there is this overarching background question of what it means for data to be internally inconsistent, and specifically what an RA should do when data is internally inconsistent. If one part of the data says one thing and another part says something else, it\u2019s easy for me to imagine an above average RA who thinks of him or herself as diligent, thinks of him herself as not needing to take big-shot professor\u2019s time with minor data glitches, who certainly doesn\u2019t want to criticize whatever prior RA did the work and created this apparent inconsistency. This above average RA would want to find a way to make the problem go away without bugging Francesca. With data inconsistency, the obvious temptation is, yes, to make the data consistent. Is it wrong? Absolutely. Is it research misconduct? It looks a lot more to me, like general sloppiness, and it looks more like what a 20-something RA might do versus what a professor is likely to do.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay. So for this part, the Hearing Committee quoted Francesca as saying, &#8220;If you want to make sure that everything is accurate in the data set and the sums are not there, you might change it so that it\u2019s consistent.&#8221; The Hearing Committee pointed to that statement and then said, &#8220;the actions in this instance were antithetical to science. Instead of fixing the sums to reflect the actual data and contacting the journal to correct the scientific record, Professor Gino manufactured the data to support the sums.&#8221; That does sound pretty bad.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Yeah, except the committee completely mischaracterized what Francesca said. The committee assumes that Francesca is describing herself, because it assumes that Francesca was the one who would put this data together to share it with Data Colada. But she was not describing herself. She was describing how someone cleaning files would have worked with the files. She had already testified that she didn\u2019t clean or prepare data files; she certainly didn\u2019t clean or prepare these data files. Thus, when she said &#8220;you&#8221; in that statement, she wasn\u2019t talking about herself. She was talking about the RAs who would have been preparing the data.<\/p>\n<p class=\"para\">Okay, so we have two radically different theories here.<\/p>\n<p class=\"para\">1. One theory is that a research assistant is asked to clean up the data to send it off to this data fraud investigator. They do some cleanups that are perfectly reasonable, like clarifying the titles of a column, and they do other cleanups that are not reasonable. One seems pretty innocent: they didn\u2019t understand why there\u2019s an inconsistency between the &#8220;reported guessed correctly&#8221; column and the &#8220;cheated&#8221; column. So they just correct the &#8220;reported guessed correctly&#8221; column, stupidly, if you\u2019re trying to hide fraud. But that\u2019s what they would have done. More troublingly, they just changed the numbers related to the elements that added up to some aggregate score for these four variables, wrong, but wrong done by a research assistant, not by Francesca.<\/p>\n<p class=\"para\">2. The alternative theory is that Francesca made these changes. The changes to the 12 participants would have been intended to hide earlier changes to 12 participants. But again, that change would have been done in the dumbest way you could have imagined if you were trying to hide the fraud. With respect to the four creativity scores, this would have been fudging to hide the fact that the creativity scores had been amplified.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So two radically different assumptions about what went on here. What would you need to know to be able to tell which of these two assumptions actually reflected reality?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">What you really need is an interview. Francesca\u2019s records indicate that there are at least five RAs who could have been involved in the process of preparing the data to be shared with the Data Colada researcher. Harvard knew the names of these five RAs. A conversation with them about preparing this data could have determined whether they had in fact prepared the data, and in preparing the data, whether they had in fact noticed the inconsistencies that the innocent account suggests they changed. That question could have at least been asked. Said, Hey, maybe one of them would have emails about this, emails that Francesca didn\u2019t keep. If it had been asked, and the research assistant said, &#8220;I don\u2019t remember,&#8221; we\u2019re in no better position than we are now. But if it had been asked, and the research assistant said, &#8220;Yes, I did that, and I probably shouldn\u2019t have,&#8221; then we\u2019ve established Francesca\u2019s innocence. If the RA said, &#8220;Yeah, I cleaned the data up. I didn\u2019t touch anything about that. I just changed the column headings,&#8221; that would again tend to support the hypothesis that Francesca was guilty.<\/p>\n<p class=\"para\">So once again, a basic investigation here could have provided the basic evidence to disambiguate between these two radically different hypotheses about what actually happened that the changes were the byproduct of a messy research assistant, or that the changes were intentional fraud on Francesca\u2019s part. Once again, Harvard failed to take the essential investigatory steps to allow that ambiguity to be resolved. Instead, they simply assumed that she must have made these changes, rather than an RA making these changes, and that assumption convinced them or allowed them to conclude she was guilty.<\/p>\n<p class=\"para\">Okay, so let\u2019s bring all of this together. This allegation number three: this is a hard allegation to understand. Multiple people were involved over more than a year, key files are unavailable, and at least some of the anomalies that were identified were indisputably created by research assistants. We don\u2019t have the data to see whether all of the anomalies were produced by research assistants. And most of all, we don\u2019t have any investigation by Harvard that looked at what the RAs said or did to evaluate whether these anomalies were Francesca\u2019s doing.<\/p>\n<p class=\"para\">Some of these questions may be answered in the subsequent investigation that\u2019s ongoing in the ongoing litigation, but whether they are answered there or not is irrelevant to the fundamental point that I want to make here.<\/p>\n<p class=\"para\">That point is this: the big claim Harvard makes is that it concluded Francesca was guilty after investigating her guilt. But to reach that conclusion, they did not do a fair or complete investigation. They went through the motions with a presumption in their head, the presumption that she was guilty, they failed in the most fundamental way to adduce the evidence that could distinguish between fraud and a mistake.<\/p>\n<p class=\"para\">Okay, I\u2019m grateful once again to my friend appearing through the voice of Ava.<\/p>\n<p class=\"para\">Stay tuned for the analysis of the final allegation, number one, which I will publish two days after I publish this episode. I\u2019m again sorry for the delay it\u2019s taken to complete this allegation. I personally had a book deadline that I had to clear before I could give it the attention it required.<\/p>\n<p class=\"para\">I am really eager to get this one out and the next one out and then one final episode that will draw together what all these suggest and where I think Harvard should go from here.<\/p>\n<p class=\"para\">Thanks again for listening. Stay tuned for the next episode. These episodes are produced by Josh Elstro of Elstro Productions. I\u2019m grateful to him. They\u2019re not funded by Equal Citizens.Us. They are funded by me.<\/p>\n<p class=\"para\">You can find this podcast where you find podcasts. Again, thanks for listening. Stay tuned for the next episode in 48 hours.<\/p>\n<p class=\"para\">This is Larry Lessig.<\/p>\n<\/div><footer class=\"ep-footer\"><a class=\"ep-footer-link\" href=\"#gino-index\">\u2191 Back to index<\/a><a class=\"ep-footer-link next\" href=\"#gino-episode-8\">Next: Allegation #1 \u2192<\/a><\/footer><\/section>\n<section class=\"episode\" id=\"gino-episode-8\" data-screen-label=\"08 Allegation #1\"><header class=\"ep-header\"><div class=\"ep-eyebrow\">Episode 08 &middot; The Law, Such As It Is &middot; Season 3<\/div><h2 class=\"ep-title\">Allegation #1<\/h2><p class=\"ep-blurb\">The only allegation within Harvard&#8217;s six-year limitation rule.<\/p><\/header><div class=\"ep-body\"><div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para has-dropcap\">This is Larry Lessig. This is the eighth episode of season three of the podcast, &#8220;The Law Such As It Is.&#8221; In this season, we\u2019ve been reviewing the decision by Harvard to revoke the tenure of Harvard Business School Professor Francesca Gino.<\/p>\n<p class=\"para\">The first episodes reviewed the procedural nightmare that brought about this result. The last three and a half have considered three of the four allegations of academic misconduct brought against her. This episode considers the fourth, what we\u2019re going to refer to as allegation number one. This is the last allegation, and after this episode, there will be at least one more episode that draws together what we\u2019ve seen across all eight.<\/p>\n<p class=\"para\">Allegation number one is the only allegation within the six-year limitation set by Harvard\u2019s rules. Therefore, from my perspective, it\u2019s the only allegation that should have been considered by the Business School and the Hearing Committee. That six-year limitation, as you\u2019ll remember, expressly says that charges of academic misconduct more than six years old, &#8220;may not be investigated.&#8221; Somebody missed that little requirement when they launched this four-part investigation four and a half years ago, an investigation that investigated three charges that were plainly more than six years old.<\/p>\n<p class=\"para\">I am joined again in this episode by my anonymized friend who we\u2019re calling Ava. And Ava will help us to unpack the charges as they relate to allegation number one.<\/p>\n<p class=\"para\">Stay tuned for this one, because there\u2019s quite an extraordinary punchline at the very end of this episode. So welcome back, Ava.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Thank you, Professor.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Ava, help us understand the context for allegation number one.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Allegation one involves a paper that Francesca Gino coauthored with two other authors. The paper was titled, &#8220;Why connect? Moral consequences of networking with a promotion or prevention focus.&#8221; It was published in the Journal of Personality and Social Psychology in 2020. In study 3A of the paper, which Francesca led, participants were randomly assigned to one of three conditions: a prevention condition, a promotion condition and a control condition. In the prevention condition, the participants were focused on meeting one\u2019s responsibilities. In the promotion condition, they were focused on growth and advancement. In the control condition, they were focused on neither to create such a mindset. Participants initially wrote an essay on hope or aspiration in the promotion condition, a duty or obligation in the prevention condition, or their usual activities for the evening in the control condition.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, but so what did they do in these three separate conditions?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">In each condition, the participants read a story about a networking event, and they were asked to imagine they were the protagonist making professional connections in the story. The hypothesis of the study was that if you were in the promotion condition, meaning you were focused on dreams and aspirations, you would have a lower sense of impurity and a greater desire to network with other people, whereas, if you were in the prevention condition, focused on obligations, that would produce the opposite results.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And I\u2019m assuming that this is a published paper, because indeed, that&#8217;s what the results produced. Is that? Right?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Yes.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so what was the basis of the charge of academic misconduct here?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Well, here\u2019s a study that was produced using Qualtrics. We once again have two files, File A and File B. File A is the downloaded data from Qualtrics, the raw data from the participants\u2019 answers. There are differences between File A and File B. In particular, there are 1,066 different values between the original File A and the final File B.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And so I imagine the charge is that Francesca made those 1,066 changes in order to strengthen the results of her paper.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Yes, that is the claim.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And is that claim true?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">No, indeed, of all four of the allegations, I see this as the weakest.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Let\u2019s start with the claim of the Hearing Committee that &#8220;all of the alterations are in the direction of the study\u2019s hypothesis, and without them, the data would not have supported that hypothesis.&#8221; Is that claim true?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Once again, that claim is just false. 39% of the 1,066 changes were of data that was not used to test the hypothesis. 39% of 1,066 is 415. So the question, once again, is why Francesca would change 415 values for no purpose at all?<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Same as it ever was.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">You\u2019re right. This has been a theme across the four allegations. The thrust of the charge against Francesca is that there were changes made to the data, and those changes supposedly all supported the hypothesis that established motive. She had opportunity, so therefore she must be guilty. But with all four allegations, the premise is false in all four papers. It\u2019s not the case that all the anomalies strengthened the conclusions of the paper.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">In this case, why did the Hearing Committee conclude that Francesca made these changes?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">The committee claims that on a single day, January 24 2020, Francesca opened the original, unaltered Qualtrics dataset sometime in the afternoon, and that she later saved a dataset based on that data, but saved that dataset with 1,066 altered values. The committee thus concluded that in those few hours, she must have made the modifications reflected in File B.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, but let\u2019s be very clear about this. What the evidence shows is just that she opened the Qualtrics dataset that afternoon, right?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Right.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">When she opened that dataset, for what purpose might she have been opening the dataset?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Yeah, that\u2019s the critical first point to realize. When you\u2019re producing data from Qualtrics to make an academic study, there could be lots of reasons for opening the original dataset. For example, you might want to check the total number of participants in the study before any exclusions that occurred at later steps that could be in your notes somewhere. It could be in the data you\u2019ve already downloaded and processed. But if you want to be absolutely certain, you go back to the original source. You open the Qualtrics dataset.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, but the Hearing Committee assumed that when she opened the Qualtrics dataset, she was opening it for the purpose of beginning the analysis of the data in order to produce her final analysis. Is that right?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Right. They thought she downloaded the data, turned it into an Excel file, opened the file in Excel, and then worked on the data to get it into a condition that she could then make the analysis using a statistical program called SPSS. And they concluded that when she did that, or while she did that, she altered the data to strengthen her conclusions.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, but I thought that she had testified repeatedly\u2014 and no one challenged this testimony\u2014that this process of data cleaning or preparing the data to be analyzed was the sort of work done by research assistants.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">That\u2019s right. Everyone agrees, and it\u2019s pretty intuitive that data cleaning was done by research assistants, not Francesca. HBS professors don\u2019t clean their own data. So if she downloaded the raw data and cleaned it up on the 24th, it would have been a first. Or at least a first since the time when she was just becoming an academic, because by this point, she had an army of research assistants helping her with her work.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Did her research assistants testify that they had, in fact, worked on this data?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Yes, that\u2019s what they said. Her research assistant at the time, Alex Rohe, testified that he had worked to clean up that data before January 24. He specifically recalled working on the data before January 24 and asked Harvard to follow up by checking his email to confirm what he had done before that date.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Did Harvard follow up on Rohe\u2019s suggestion?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">We don\u2019t know. There\u2019s no evidence that they did. All we know is they didn\u2019t introduce any other evidence about his interactions with Francesca, and didn\u2019t give her any access to that information. But the key point, as I see it, is that there was uncontested testimony that someone other than Francesca had done the boring, tedious, error prone, worked of cleaning up the data, and had done that work before January 24.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Which would mean that the file that was opened on January 24 (what we\u2019ve called File A) would not have actually been the original file from which Francesca would have done the work that led to File B. Is that right?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Right.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Did the Hearing Committee have this testimony before it?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">They did.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Did the Hearing Committee accept that Alex Rohe did, in fact, work on the data prior to January 24?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">No. The Hearing Committee rejected the suggestion of the data that was worked on January 24 was data that had already been prepared by research assistants before that date \u2014 Alex Rohe or anyone else.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Why?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">The committee gave two reasons. First, that there was no evidence that a research assistant was working with the data, &#8220;at this late stage&#8221; \u2014 meaning on January 24. Second, because no research assistant had the opportunity or incentive to alter over 1,000 data points in the direction of the study\u2019s hypothesis.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">But regarding the second reason, we\u2019ve already noted that not all the changes were in the direction of the study\u2019s hypothesis. 415 of the 1,066 anomalies did not strengthen the conclusion of the paper at all.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Right, so that\u2019s an example of the committee assuming the conclusion.<\/p>\n<p class=\"para\">But focus on the first point: that there was no evidence that a research assistant was working with the data at this late stage.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Is there evidence that a research assistant was working with the data at this late stage?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Your question kind of misses the point. While there was no evidence of a research assistant working with the data on January 24, there was testimony that a research assistant was working with the data before January 24. That research assistant was Alex Rohe, and that evidence is perfectly consistent with what would have been the normal practice for Francesca working with data: the RAs downloaded the raw data; the RAs cleaned the raw data, such as ensuring that the essays participants submitted were valid; and then the RAs put the data into a form that Francesca could analyze. Francesca testified that an RA would typically give her a file on a thumb drive or sometimes by email.<\/p>\n<p class=\"para\">So while there\u2019s no evidence that the RAs were working with the data on January 24, they didn\u2019t need to work with the data on that date. They could have cleaned the data and prepared it and given it to Francesca sometime before the 24th and then she finally gets around to working with it on the 24th.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so this alternative hypothesis that a research assistant had been working with the data to get it into a form that Francesca could work with in the time leading up to January 24; on January 24, Francesca would have taken the data that the research assistant had been working with and then worked with it to produce File B. That alternative hypothesis would imagine that there was another file that Francesca might have been working from or with from January 24. Is there evidence or a record of a file other than what we\u2019re calling File A that Francesca might have been working with on January 24 that might already have had the data cleaning done?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Yes, there is evidence that there was a file that Francesca had on her computer on January 24 that already had the data cleaned and that she would have been working from.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">What was that evidence?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">The evidence showed that there was a file in Francesca\u2019s computer, which was likely the data file with the letter &#8220;R&#8221; in the name of the file. That letter matches the last initial of the research assistant, Alex Rohe. That file does not survive in the record, I guess meaning someone moved it or renamed it, and it can\u2019t be found on her hard drive in the place where it might be expected. But that alternative file could have been the source of the data that was actually analyzed on January 24.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so this sounds pretty compelling. Again, remember, the committee\u2019s theory is that Francesca downloaded the data, the raw Qualtrics data, on January 24, something she typically never would have done\u2026 was then to take that data and to go through the tedious process of cleaning it, to put it into a condition that it could be studied or worked with. And then she worked with it and modified it, tweaked it to get to File B, but this process just literally just never happened with Francesca, because Francesca didn\u2019t clean data.<\/p>\n<p class=\"para\">The alternative theory, by contrast, is consistent with her normal practice: taking data cleaned by an RA and working with it. On that theory, she used the file that was named in a way to indicate that it was the data for the study and it had the letter R on it, that would have indicated her research assistant had worked on it. That she opened that file and proceeded to analyze the data in that file. We don\u2019t have that file. We don\u2019t know whether that file included the 1,066 anomalies or not. And so, once again, as with allegation two and allegation three and allegation four, we have no foundation for saying that she made any of these changes at all.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Right, those are the two possible alternatives. But the Hearing Committee rejected the second alternative, which seems to me the more likely alternative, given that she says she never cleaned data herself, and I don\u2019t see a lot of HBS faculty cleaning their own data.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So why did they reject this second alternative?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">The Hearing Committee found that it wasn\u2019t likely that the letter R in the file name referred to Alex Rohe because they found that there was another file elsewhere on her computer, that was also a data file also with the letter R on it. See, that other file was created before Alex Rohe was a research assistant. From that, the Hearing Committee inferred that adding the letter &#8220;R&#8221; to the file was not an indication of the name of the RA who worked on the file.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Oh, okay, so what\u2019s wrong with that argument?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Well, this again reveals how the committee didn\u2019t know their own record. At the time of this other file with the letter R, there was another research assistant working for Francesca whose last name also began with the letter R, Mindy Rock. Yes, R for Rock. Rock was working with Francesca at the time that another file was created, so this file with R actually offers support for the hypothesis then Francesca at least sometimes named files with the RA\u2019s last name initial.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Wow, so the Hearing Committee basically rejects the more likely alternative based on an inference that itself was based on a factual mistake.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Right. They reject the idea that R referred to the research assistant actually working with Francesca at the time because there was another file with the letter R that didn\u2019t refer to that same research assistant\u2014apparently not recognizing that other RAs might also have last names that begin with the letter &#8220;R.&#8221;<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, but did Alex Rohe testify that he had been working on data and had sent it or given it to Francesca? Just to be clear about this point again\u2026<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Yes, and this is what we discussed a few minutes ago. That\u2019s the craziest thing about this allegation. Alex Rohe \u2014 Francesca\u2019s main RA at the time \u2014 was one of just two research assistants that was actually interviewed during the HBS investigation. He specifically testified that he was working on the data file and implied that he had given her a copy of the data that he worked on. All that he did before January 24. The Hearing Committee thus had evidence from somebody who actually worked on the paper that he had actually produced at least a partial version of the data and had given it to Francesca before January 24.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And so if he did that, that means that on January 24, when Francesca was deciding to actually complete the analysis of the paper, she could either have taken the data already cleaned by her research assistant and worked from that, or she could have re-downloaded the data, the raw data, and on her own, cleaned it and prepared it to be analyzed. Is that right?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Right. They had those alternatives before them\u2014the completely ordinary process that would have suggested the data she worked with was Alex Rohe\u2019s modified version of the data, worked and cleaned version of the data, and the completely extraordinary process that she re-downloaded the data to clean or reclean it herself\u2014they concluded that she did the latter, which seems to me, plainly the less likely of the two. And on the basis of that conclusion, they concluded, supposedly with &#8220;clear and convincing evidence&#8221;, that she\u2019s committed academic fraud.<\/p>\n<p class=\"para\">Wow. Seriously, just wow. They had actual evidence from a research assistant that he had worked on the data, and on the basis of a mistaken inference about a naming convention for files produced by RAs, they reject that more probable scenario and instead adopt the less probable scenario on the way to concluding what is even a less probable outcome: that Francesca modified 1,066 values, including 415 values that had no relation to the ultimate strength of the hypothesis as an act of academic fraud.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, but didn\u2019t the hearing committee also conclude that &#8220;she analyzed these two files over the course of an afternoon using SPSS running commands in a manner consistent with repeatedly altering the data and then checking whether it improved the results&#8221;? Because that sounds pretty bad.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Yes, the Hearing Committee concluded that.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Well, that sounds bad. Is it true?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Sure, it sounds bad, but it\u2019s also not true. The finding is speculation refuted by the actual logs created by that SPSS session.<\/p>\n<p class=\"para\">When examined closely, these logs demonstrated that she ran many commands across four different data sets supporting three different studies. She had them all open throughout the SPSS sessions, running these different commands as she was working across these different studies.<\/p>\n<p class=\"para\">The SPSS logs thus do not demonstrate that she reran the same commands over and over in the way that the Hearing Committee claims to have found. And any duplicate commands were caused by switching between the files she was analyzing; analyzing filtered sets or subsets of the data; closing and reopening SPSS; and by the efficiency of rerunning a command rather than scrolling up in a log to find the results from that command. Again, some of the commands appear similar on the surface, but actually have different controls or different specifications for the statistical model used. Once the command differs even a little, the best interpretation is data exploring \u2013 &#8220;what result do I get if I analyze the data this way or that way?&#8221; or some modification of this, not the data tampering and retesting that the Hearing Committee claimed to have found.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay. But how did the Hearing Committee reach its conclusion in light of this evidence, assuming that they could read SPSS log files?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Once again enter Professor Freese.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Oh no, not Freese again.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Yes. Professor Freese. The Hearing Committee based its conclusion on a statement Professor Freese made with great confidence in his report. From the SPSS log, Freese claimed that it &#8220;shows that Gino\u2019s work with the data involved her repeatedly opening Excel data for the study, computing some results that she had already computed, and then reopening the data file.&#8221; This behavior, Freese suggested, was very suspicious. It suggested the method by which Francesca would have committed academic fraud on Freese\u2019s account. Francesca would have opened a data file and tested it in SPSS. She would have seen the results, then gone back to the data file and tweaked the data. Then she would have tested it again with SPSS with the same exact command, and if it still needed more tweaking, she would have gone back to the data file and changed some more.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">That\u2019s pretty clever. It\u2019s a pretty good way to fudge just enough to get the results you need. But on this theory, why did she change the extra 415 cells of data that had no effect on the result?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">This theory can\u2019t explain that. But that\u2019s not even the strongest evidence against Freese\u2019s hypothesis.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">What\u2019s better than that?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">The actual SPSS log, showing precisely which commands were run during these sessions. On Freese\u2019s theory, it was the same command being run on the same data over and over again. But the actual SPSS log shows that is just not what happened.<\/p>\n<p class=\"para\">I spent time examining the SPSS log myself. It consists of commands resulting in about 63,000 lines of code. The interaction between SPSS\u2019s menu and log file is subtle. Many SPSS commands dump large blocks of code into the log file. Some analyses, like the mediation analysis used in the study, create hundreds of lines of code in the log from just a couple clicks in the menu. But, okay, you can take out these huge blocks and look at the actual commands that Francesca actually issued.<\/p>\n<p class=\"para\">Inspecting the log carefully, I found that it contains commands from three separate SPSS sessions, meaning closing and reopening all of SPSS three times. The three SPSS sessions occur at different points in time, at about two in the afternoon, then about nine at night, and about 10:30. There are 15 fileopen commands relating to four different files, and a total of 61 distinct data analysis commands across the four files in three sessions.<\/p>\n<p class=\"para\">On some level, I understand Freese\u2019s hypothesis about reanalyzing the same file. But here\u2019s the thing: only six of those commands, six of 61, occur more than once for the dataset at issue across the three SPSS sessions \u2013 four of them twice, two three times. None more than that. And in one of these instances, Francesca ran the same UNIANOVA twice with identical syntax, one time right after another, without reopening the data set in between the two times. If she was changing the file and testing whether the change had the statistical impact she wanted, she would have needed to actually reload the file in order to get the changed version.<\/p>\n<p class=\"para\">Freese does not engage with this fact, which is flatly contrary to his hypothesis of modifying the data in another program and reloading it into SPSS. His theory does not explain why Francesca ran the same command twice in a row without any reloading. The obvious simple hypothesis that she was exploring the data and trying to understand what it showed explains this behavior perfectly. Another natural hypothesis is that she\u2019s not some perfect villain whose every command perfectly achieves some evil objective. No, she\u2019s more like a normal user, kind of muddling around a program she basilittle confused, maybe cally understands, but also a distracted, including innocuously rerunning the same command. Because really, that\u2019s just not a big deal. It\u2019s not like we\u2019re paying by the command here.<\/p>\n<p class=\"para\">Francesca also ran one UNIANOVA on a key measure only once throughout the afternoon session, again contradicting Freese\u2019s theory that she spent time that afternoon changing the underlying data behind that measure.<\/p>\n<p class=\"para\">When Francesca ran different specifications for the same analysis, Freese categorized them as one identical analysis repeated over time. But that\u2019s wrong too. If a person were modifying the data in another program and then checking in SPSS whether the modification mattered, the SPSS command the person would run would be exactly the same each time. But Francesca didn\u2019t run exactly the same command each time. Instead, she ran slightly different commands, performing different analyses with each command, which again makes sense for someone trying to understand what\u2019s in the data, but it\u2019s flatly not what you would do if you were testing whether your modifications of the data have achieved your target results.<\/p>\n<p class=\"para\">So again: the logs showed that there were different commands runs, meaning that Francesca was not changing the data in a separate program and testing whether that improved the results. I think Freese knows how to read the SPSS logs, but he came into this looking to find evidence that Francesca was guilty, and his whole report leads down that path, not engaging with the evidence within these very same logs, that pretty well points in the opposite direction.<\/p>\n<p class=\"para\">I think we all know that busy researchers and important professors get interrupted. Francesca\u2019s calendar for the day shows she had an office meeting and a child to pick up, but within the general period of the timestamped SPSS afternoon analysis. Freese assumed that all data analysis would be done in one continuous session with no reloading, but that\u2019s really not consistent with what the calendar says she did that day. Now, the SPSS log does not record the time of individual commands, just the start of each session and the order of commands within each session. Still, the logs show scattered work during the business day, consistent with a person having meetings, forgetting what she was doing, coming back to it and redoing some work she had already done. And then late at night, surely, after the kids were asleep, the logs show somewhat more purposeful work overall, in an order matching what\u2019s in the paper.<\/p>\n<p class=\"para\">Francesca\u2019s innocent explanation thus matches what you\u2019d expect from a busy professor and someone with young kids. Her youngest was two and a half months old at the time. Freese imagines an evil genius tweaking data\u2014 well, there\u2019s just not support for that in the SPSS logs.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So it sounds like both you and Francesca\u2019s forensic expert disagree with what Freese and the Hearing Committee concluded after examining the SPSS log.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Yes, the Hearing Committee\u2019s conclusion from this data is refuted by an informed interpretation of the SPSS log file.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so pulling this together. Basically, we have two experts: Harvard\u2019s data expert, and Francesca\u2019s forensic expert. Harvard\u2019s data expert, Professor Freese, said she downloaded the data and cleaned it and then made the changes that produced File B. Francesca\u2019s expert said she worked from a file that had been created by somebody else, and so we don\u2019t know whether the differences between the raw data and the final product are the product of her modifying it, or whether the anomalies were created by the research assistants.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Well, no, it\u2019s even more outrageous than this. Because this was a rare instance when both the Business School\u2019s forensic expert and Francesca\u2019s forensic expert actually agreed.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Wait, I thought you said Freese was Harvard\u2019s expert.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Yes, Freese was the data and statistics expert who came into the case at the very last minute to support removing her tenure. But there was a second expert who HBS brought into her tenure case: a forensic expert.<\/p>\n<p class=\"para\">Both HBS forensic expert and Francesca\u2019s forensic expert testified at the tenure revocation hearing that they believed Francesca did not work from data downloaded from Qualtrics on January 24. Both experts believe that Francesca worked from data that she had copied and pasted from some other file, specifically not, as the Hearing Committee concluded, from the Qualtrics data.<\/p>\n<p class=\"para\">Thus both experts are effectively agreeing that the real file that she was working with beginning on January 24, was not the one downloaded from Qualtrics, but some file created before January 24. And again, Francesca argues that that file would have been created by an RA, most likely Alex Rohe.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So, what basis did the Hearing Committee have for concluding that she downloaded the data on January 24, cleaned it herself, modified it, and with that modification, eventually producing File B?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">No evidence. Literally none. It was a fabrication based on Freese\u2019s hypotheses about what must have happened. Hypotheses driven by their conclusion that she was guilty. She must be guilty, that this is the cleanest way to explain how she was guilty. She started from a clean, raw data file, did what she never did in the process of working with data, namely clean the data herself, ignoring the work she had hired a research assistant to do, work which the research assistant testified he had done prior to January 24, and then she manipulated that data to produce the anomalies that are reflected in File B.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so I take it you would then say that there\u2019s not clear and convincing evidence that she modified the data to produce File B.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">There is not clear and convincing evidence. There\u2019s not even more likely than not evidence. There\u2019s pretty close to no evidence at all. There\u2019s no evidence in this case to support the conclusion that she downloaded the data on January 24 and manipulated that data herself to produce the anomalies that are reflected in File B.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">But if she didn\u2019t manipulate the data, what explains the 1,066 differences?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">I spent some time exploring the data to try to answer that question. What I discovered is that there are good reasons to think these were errors, not falsifications. Even without access to all the evidence we would ideally want, 62% of the changes can be explained by just one of three known types of data processing errors.<\/p>\n<p class=\"para\">This one error is an Excel &#8220;all-to-one&#8221; error. It happens when a user copies and pastes, but selects a range rather than a single cell, arranges like group of cells when giving the Paste command. If you paste a single cell into a range, the Paste command repeats the cell\u2019s value across all of the cells of the selected destination range. If the RA had committed that error, then 62% of the changes would be accounted for by that alone. You know, it\u2019s sort of a weird thing to do. If I wanted to paste a cell into a cell, I select the one source, copy, select the one destination, paste. But if someone for some reason has a different habit, like has a habit of selecting a group of cells in the supposed destination and does that over and over, yeah, you could change a lot of data. You could override a lot of data that way, maybe without even realizing it.<\/p>\n<p class=\"para\">I also examined how other errors can explain some of the anomalies. Errors like a flip error, where values coded on a 1\u2013 7 scale are incorrectly reversed, like seven to one during recoding. That\u2019s an easy error for RAS to make, and that error seems to be present in this data too.<\/p>\n<p class=\"para\">Or, the study in Qualtrics may have initially assigned arbitrary numeric coding to Likert responses.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">The Likert responses?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Yes, these are the responses that measure the intensity of feelings like, strongly agree to strongly disagree. These labels make sense to people, but the data analysis requires sequential numbers. That means the data would have to be recoded in Excel or Qualtrics, depending on when the RA realized that numbers are needed and the numbers could be messed up.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Messed up?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Yes, because apparently Qualtrics sometimes converts the logically-ordered English labels into out of order numbers. You would think like, strongly agreed was one. Slightly agreed was two. Neutral could be three. But apparently sometimes Qualtrics changed those numbers so they\u2019re not in order, and at that point you have to put them back in order. It\u2019s strange. I don\u2019t know why Qualtrics would mess that up, but I read multiple online threads of other users complaining about it. So we need some information from Qualtrics to determine what role this error might have played and whether this error was likely to have occurred in this dataset.<\/p>\n<p class=\"para\">These are the kinds of mistakes known to occur when cleaning and formatting large datasets. You know, stuff happens. Computers are complicated, multiple systems connecting\u2026 And as I say, just the first of these errors, the all-to-one, covers 62% of the data anomalies.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Still, though, that\u2019s a lot of speculation.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">It is speculation. But the point is, it\u2019s a way of understanding how the results we see could have been produced while the data was being prepared by the RAs rather than assuming, and they did assume, that Francesca was a fraud.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">But it\u2019d be really great, wouldn\u2019t it, if we had some way actually to see what files she had opened or what files she had used on January 24. I mean, it would be really great if there was a spy that had been sitting in the room with her that could tell us what exactly she did on January 24. Because if there\u2019d been that spy, that spy could tell us whether it was this lost file from Alex Rohe that she had opened on January 24 or whether she had, as the Hearing Committee found, downloaded the data herself, cleaned that data and then done that manipulation of the data in order to produce File B.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">What do you mean, we should have had that spy?<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">I mean, it would be really great to have a spy in the room like that, right?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Okay, but you know, we did have sort of a spy. There was a spy sitting in the room watching which files were opened on her computer. And that spy such as it is, it\u2019s the computer\u2019s operating system. The computer itself recorded which file she opened that day.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">What do you mean by that?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Well, the log files of the computer of the operating system would have recorded, for example, that she inserted a thumb drive and opened a certain file from that thumb drive, or if she downloaded data at a certain time from the Internet.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Wait\u2026 so the log files of the operating system of the computer she was working on would have captured and reported what actually happened that day. Did Francesca or her experts look at that log file?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Well, her team tried to get them. They asked Harvard to supply the log files so they could see exactly what happened on that day, exactly which files were opened.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And what did Harvard say to their request?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Harvard said, we don\u2019t have them.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Why didn\u2019t Harvard have them?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">It appears Harvard\u2019s investigators made maybe the most basic forensic mistake that could have been made in any forensic study involving a computer.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">What mistake was that?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Well, hold on to your chair here, because this is really going to blow your mind.<\/p>\n<p class=\"para\">When the Business School opened the investigation in October 2021, the first thing the Research Integrity Officer did was to seize Francesca\u2019s computer. Francesca asked them whether they had taken an image of the computer at the time they seized it, and in an email dated November 4, 2021 the RIO told Francesca that he had made such a copy of her hard drive: &#8220;All the electronic files included in the inventory are forensic copies, and the original sources remain available to you.&#8221;<\/p>\n<p class=\"para\">Forensic copies. But 26 months later, when Francesca\u2019s forensic expert received the evidence from HBS, he discovered that, in fact, no forensic image of Francesca\u2019s computer had been made.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">What do you mean by an image? A picture?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Not exactly, but sort of. A full copy of the entire hard drive. It\u2019s Forensics 101 that when you\u2019re doing a forensic analysis involving a computer, you basically take a snapshot of the hard disk of the computer. Not just the files that normal users access, but every single file on the computer, including system files and log files. And you do that at the start, immediately, because any delay could cause logs to be overwritten.<\/p>\n<p class=\"para\">So if they had taken a snapshot, a full forensic image when the computer was seized, it could have been possible to determine exactly what happened on the day when the files were on that computer. You could have determined whether a thumb drive had been inserted into the computer, whether a file had been read from the thumb drive, which was Francesca\u2019s theory of what happened, or whether there was no thumb drive inserted that day, and instead the data that produced the file that actually became File B was downloaded from the Internet, as Harvard alleged.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So why didn\u2019t Harvard have that image? Did they lose it?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">No, the Keystone cops that ran this investigation never took an image of the computer\u2019s hard drive.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">What do you mean, they never took an image of the computer\u2019s hard drive?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">I mean exactly that they never took an image of the hard drive. What they did instead is they had a technician copy some files from Francesca\u2019s computer onto a separate drive. Not all the files, just the particular files that the RIO thought might have been involved in the research at issue. They made no effort to copy everything. They didn\u2019t use imaging software.<\/p>\n<p class=\"para\">On some level, well, I guess I sympathized with the technician. What training did he have about what it meant to take a &#8220;image&#8221;? And did he have the right tools to do that? But HBS is a big and wealthy place. Boston has great tech talent. If they wanted this investigation done properly, they could have found someone to do it properly. But\u2026 they didn\u2019t. By the time this blunder was discovered \u2014 all these log files, of course, had been replaced with new log files, as the operating system, and its standard practice, goes out with the old and in with the new. Don\u2019t want to keep gigabytes of logs around forever, so it really does make sense for the operating system to get rid of the old logs. But there was no way to go back to see exactly what happened on that day, because by the time Francesca\u2019s lawyers discovered Harvard\u2019s mistake, the operating system had erased the log files.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so this is a little mind boggling. You\u2019re saying that if the business school had conducted their investigation in the way that any forensic investigation of technology would have been conducted, if they had gone into her office and taken an image of the entire computer disk at the time the investigation started, the image could have the evidence necessary to distinguish between two fundamentally different hypotheses:<\/p>\n<p class=\"para\">Hypothesis 1: That she had taken data given to her by her research assistant, data that had already been cleaned and prepared for her to analyze, and then analyze that data, producing File B, or<\/p>\n<p class=\"para\">Hypothesis 2: That she had downloaded the data from Qualtrics, and cleaned that data and prepared and therefore had done the manipulation herself.<\/p>\n<p class=\"para\">That evidence could have been preserved in the image taken of the computer disk, but Harvard didn\u2019t have that evidence because Harvard\u2019s inept investigation had not understood that when you\u2019re investigating a computer incident, you take an image of the entire computer disk. You don\u2019t copy a few folders of files.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">That\u2019s right. HBS failed the most basic test of computer investigations. And that failure denies both HBS and Francesca the evidence to distinguish between these two hypotheses. Now, my view is that these two hypotheses are, on their face, so different in probability that you don\u2019t exactly need that forensic evidence\u2026 but still, that evidence would have resolved this question with certainty.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">And we don\u2019t have that evidence again because of Harvard\u2019s own basic ineptitude.<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">That\u2019s right. The incompetence of the HBS investigation denied Francesca the chance to prove what seems clear, even without this evidence: that whatever explains the gap between the Qualtrics data and the file that she used in the analysis, the explanation is not that she intentionally modified the data. Whatever modifications, whatever anomalies, whatever reasons there were for the differences between the raw data and File B, the evidence Harvard claimed to have preserved but didn\u2019t actually preserve, would surely have been helpful in her effort to prove that this was not research misconduct.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so maybe if you\u2019re a prosecuting University and you realize that your own complete incompetence has denied the person whose career you are destroying the opportunity to establish your innocence, maybe at that point you say to yourself, &#8220;Oops, sorry, we screwed up. And because we screwed up, the evidence in this case is incomplete. And it\u2019s incomplete in a particular way, in a way that fundamentally weakens the defendant\u2019s opportunity for a defense that\u2019s on us, and we therefore refuse to proceed with the prosecution on the basis of this allegation.&#8221; You would think that\u2019s what they would say. So is that what they, in fact, said?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">No, they did the opposite of owning that incompetence.<\/p>\n<p class=\"para\">In fact, what they did is even worse. At the tenure revocation hearing, in November 2024, the RIO blamed Francesca for this mistake. He claimed that he asked Francesca what files to copy because he was worried about her privacy. Yet, the subset of files that he actually copied included a folder called \u2018Personal,\u2019 which included her tax returns. If he was actually concerned about her privacy, he certainly would not have copied those files. And anyway, he told her in an email he had made a &#8220;forensic copy.&#8221; Forensic. That\u2019s a technical term with a technical meaning. Copying some files from some folders is kind of the opposite of forensic.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, wow. So HBS incompetence not only destroyed this evidence, but then they blamed her for it.<\/p>\n<p class=\"para\">Okay, so there\u2019s one final bit to this charge related to allegation number one. The committee found that Francesca had &#8220;already drafted a description of the study results, suggesting she had altered the data to conform to her pre written draft.&#8221;<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Yeah, this bit of the charge is even more bizarre. The claim is that the Hearing Committee found a file on her computer that was drafted before the data analysis had been completed \u2014 before January 24 \u2014 that described the results that the analysis ultimately supported without the details, such as the means, stats or p-values. That supported the claim, the committee found, that she must have modified the data to conform it to this description written in advance.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">Okay, so had the Business School made this charge against her originally?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">No, they didn\u2019t. They didn\u2019t make this charge because the practice that the Hearing Committee flagged is completely normal within the field that Francesca was working in.<\/p>\n<p class=\"para\">In co-authored papers, in describing and designing the study, the authors often draft a version of the conclusions to decide whether the paper is actually worth pursuing. That\u2019s especially true when a paper might include multiple studies and you have to decide which of them, which combination, is worth prioritizing. Frankly, it\u2019s just prudent. There\u2019s no point in conducting a study that, if successful, would yield a finding that gets a shrug from the field. So, the fact that there was a draft describing the conclusions that eventually were produced is a complete nothing burger.<\/p>\n<p class=\"para\">Some people like to dump on Francesca\u2019s field. It\u2019s not necessarily the very fanciest in methodology, but in writing up the finding in advance, they\u2019re really not alone. For example, Amazon trains its employees when designing a new product or service to write the press release for that product or service before they write the first line of code. If the press release is lame or hard to write, probably the product will be a flop. For Francesca and co-authors, the finding is the product.<\/p>\n<p class=\"para\">So the committee drew its conclusion that this early draft was evidence of fraud based on a completely undeveloped record\u2014undeveloped because no one had ever suggested that there was anything wrong with his practice. And had they done so in the Business School, at the stage of the investigation in the Business School, it would have been dropped immediately: Francesca could have pointed to 50 other scholars who similarly write their conclusion up before they do the research to check whether a study is worth pursuing. Instead, it was the Hearing Committee\u2019s wild speculation that they had found a smoking gun, and they then deployed this as yet another way to support a conclusion that they had obviously started with: that Francesca was guilty.<\/p>\n<div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para\">So that\u2019s allegation one, right?<\/p>\n<div class=\"speaker-tag sp-ava\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Ava<\/div>\n<p class=\"para\">Right.<\/p>\n<p class=\"para\">Okay, let\u2019s just pull it together. The essence of allegation one is that there are two files, File A and File B. File B is different from File A. That difference is captured in the 1,066 anomalies.<\/p>\n<p class=\"para\">The Hearing Committee believed that those 1,066 anomalies were produced by Francesca. They found that all of those anomalies strengthened the results, and therefore that was the evidence of fraud. On that basis, they declared her guilty, but as with the earlier allegations, the claim that the anomalies all strengthened the results is just not true. Harvard says it\u2019s true, but if you look at the actual changes and the statistics the paper reports, it is not true. This undermines their motive theory absolutely.<\/p>\n<p class=\"para\">Also, as with the earlier allegations, the foundation for the claim \u2014 that File A is the file Francesca worked from \u2014 is equally wrong. Not only would it have been extremely improbable that Francesca would have downloaded the Qualtrics data and cleaned it herself, there was testimony that a researcher had produced the data that was analyzed by Francesca on January 24 and given it to her before January 24 in the standard way in which research assistants shared data with Francesca at that time, meaning on a thumb drive.<\/p>\n<p class=\"para\">If the data from her research assistant, Alex Rohe, had been the data that she had worked from on January 24, then to establish she had committed fraud, we would have to compare that data, that file to File B, and any anomalies between these two might be the foundation for a claim of academic fraud.<\/p>\n<p class=\"para\">But we don\u2019t have Alex Rohe\u2019s file. We only have a reference to it on the computer, so there\u2019s no foundation for the claim that she had modified any data at all.<\/p>\n<p class=\"para\">And worse, the incompetence of the university\u2019s investigation likely destroyed the evidence that could have established the source of the data that she began working with on January 24. If Harvard had done what the Research Integrity Officer said had been done, if he\u2019d actually produced a forensic copy of the computer at the time the investigation started, the log files on that copy could well have revealed what files were opened from what source on January 24, resolving finally, whether it was a thumb drive that had provided the data or data downloaded from Qualtrics on the internet.<\/p>\n<p class=\"para\">We don\u2019t have that evidence. What we have is:<\/p>\n<p class=\"para\">1. The ordinary practice that she would never have cleaned data herself.<\/p>\n<p class=\"para\">2. Both forensic experts agreeing that the source of the data she analyzed on January 24 was copied from some other file. And<\/p>\n<p class=\"para\">3. Her own research assistant testifying that he had provided data, a file, to her that he had already done at least partially the cleaning of the Qualtrics data.<\/p>\n<p class=\"para\">Add those together, and again, it\u2019s not just that you don\u2019t have clear and convincing evidence of her guilt. It\u2019s not just that you don\u2019t have more likely than not evidence of her guilt. Add those facts together, and you have clear and convincing evidence that there is no foundation for finding that she made any modifications at all.<\/p>\n<p class=\"para\">An integrity-driven prosecution of Francesca would have confessed this error and dropped the prosecution. But that\u2019s not what Harvard did. Harvard tripled down.<\/p>\n<p class=\"para\">That\u2019s the end of allegation number one, which means that\u2019s the end of all four allegations that we\u2019re covering in this season of the podcast.<\/p>\n<p class=\"para\">In the next episode, I\u2019m going to pull all this together so you can be reminded of where we\u2019ve been and have a concise way to understand the essence of the defense, the defense to establish that in fact, Francesca Gino did not commit academic fraud.<\/p>\n<p class=\"para\">But I want to end by thanking my anonymous friend. Anonymously. No one would believe just how much work this friend has done to help understand the facts in this case and unpack the arguments. Effectively, all of that work done pro bono, which for the non-lawyers on the line means all of that work done for free. It\u2019s rare to find such talent in general. It\u2019s especially rare to find such talent willing to devote their talent simply to demonstrate a simple truth that Francesca Gino is innocent.<\/p>\n<p class=\"para\">Stay tuned for the final episode. This is Larry Lessig.<\/p>\n<\/div><footer class=\"ep-footer\"><a class=\"ep-footer-link\" href=\"#gino-index\">\u2191 Back to index<\/a><a class=\"ep-footer-link next\" href=\"#gino-episode-9\">Next: Closing Argument \u2192<\/a><\/footer><\/section>\n<section class=\"episode\" id=\"gino-episode-9\" data-screen-label=\"09 Closing Argument\"><header class=\"ep-header\"><div class=\"ep-eyebrow\">Episode 09 &middot; The Law, Such As It Is &middot; Season 3<\/div><h2 class=\"ep-title\">Closing Argument<\/h2><p class=\"ep-blurb\">A closing argument: drawing together everything across the prior episodes.<\/p><\/header><div class=\"ep-body\"><div class=\"speaker-tag sp-larry\"><span class=\"speaker-dot\" aria-hidden=\"true\"><\/span>Larry<\/div>\n<p class=\"para has-dropcap\">This is Larry Lessig.<\/p>\n<p class=\"para\">Over the course of almost a year now, I\u2019ve been developing this podcast to unpack the charges made against Francesca Gino in Harvard\u2019s determination to remove her tenure as a professor in the Harvard Business School. Francesca is the first Harvard professor in the whole of Harvard\u2019s history to have her tenure revoked. As I said at the very start and throughout this podcast, it\u2019s my belief that Francesca is innocent of the charges made against her. Not just that Harvard has failed to prove its case or prove its case according to the standard it must prove, but that she did not engage in academic fraud. Period.<\/p>\n<p class=\"para\">It\u2019s taken a long time to complete the review, and I apologize for that. This isn\u2019t my job. I\u2019ve got too many jobs. In addition to the actual job I have, which is as a law professor at the Harvard Law School, I\u2019ve also been struggling mightily to complete a new book. That book is now at the publisher, so I have a couple of weeks to breathe, and that has allowed me to complete the last two allegation episodes and this final episode, or at least final episode for now.<\/p>\n<p class=\"para\">My aim in this episode is to draw together what we\u2019ve seen across the earlier seven episodes. (The first episode was just an introduction.)<\/p>\n<p class=\"para\">Think of it as a kind of closing argument. I\u2019m going to remind you, if you\u2019ve actually listened from the start of this podcast, of the arguments of the case. And I want to leave you with a clear sense of the flaws and a clear sense of the basis for my belief that Francesca is actually innocent.<\/p>\n<p class=\"para\">This won\u2019t be a short episode. I apologize for that. I\u2019ve added some breaks. Obviously, you\u2019re free to take whatever break you\u2019d like. And if you go to the website, the GinoCase.info, you\u2019ll see we\u2019ve set up an AI that gives you access to the transcript of all of the episodes of this podcast, and a cool little tool that you can use to ask the AI to interrogate the record that we\u2019ve put in the AI to answer whatever other questions you might have. Okay, so let\u2019s go.<\/p>\n<p class=\"para\">What went wrong here? I don\u2019t actually believe that Harvard had it out for anyone. Certainly, it didn\u2019t have it out for one of its star academics. So how should we understand how Harvard let things go so wrong? And what should happen now?<\/p>\n<p class=\"para\">As I\u2019ve thought about this case now for almost three years, I think this whole disaster begins with an understandable panic. The dean of the Harvard Business School was essentially threatened. Data Colada, a group of data scientists who\u2019ve taken it upon themselves to police academic research and call out data fraud, wrote to the business school in July 2021, and told the business school that they believed Francesca was guilty of academic fraud. And that unless the business school acted, Data Colada would publish their results. And in a not-so-veiled threat, Data Colada sent to HBS now available in the litigation docket, Data Colada indicated that their sources, if not satisfied, would, &#8220;take matters into their own hands.&#8221;<\/p>\n<p class=\"para\">Now this podcast has not been about Data Colada. As I said at the very start, I\u2019ve been an admirer of Data Colada\u2019s work, and I certainly believe it\u2019s important that there be an effective check on academic fraud.<\/p>\n<p class=\"para\">But what was striking about Data Colada\u2019s threat against the Harvard Business School dean, is that they proceeded contrary to their own stated principles. On their website, they say that this, and this is a quote, &#8220;our policy is to share drafts of blog posts that discuss someone else\u2019s work with those authors to solicit feedback seven days before the posts go live.&#8221; I agree with that policy to give their targets a chance to respond first, it\u2019s good that they do that. They have smart reasons for doing that. Their work is better for it. But they didn\u2019t do that with Francesca. They didn\u2019t write to her and tell her the basis for their concerns. They instead went directly to her boss. And apparently the dean, terrified by the idea of nasty words on Twitter, launched an extraordinary process to determine whether, indeed, Francesca was an academic fraud.<\/p>\n<p class=\"para\">And by extraordinary, I mean literally extra-ordinary: beyond the ordinary. Because the business school already had a procedure for addressing questions of academic misconduct. By all indications, that procedure was hammered out by the business school faculty, because that\u2019s the standard approach the business school used when introducing new policies affecting faculty. It provided the target of an investigation with certain process rights. It was a standard, regular procedure for determining whether, in fact, someone has committed academic misconduct.<\/p>\n<p class=\"para\">That procedure, however, was apparently not specific or efficient enough for the dean, or at least the dean tweaked by Data Colada. So the extraordinary procedure that he launched changed it. The dean crafted a new way to investigate academic misconduct, and he launched it in this case, subjecting Francesca to its new terms. The dean never brought this procedure to the business school, never asked the faculty to approve it, and never told anybody outside of his own administration that he had devised a new way to determine whether someone was indeed guilty.<\/p>\n<p class=\"para\">But before I describe the new procedure, let me just remind you of our basic timeline and the stages in this process, so that when I\u2019m describing the details below, it will be at least as clear as it can be what exactly I\u2019m talking about. I\u2019m going to divide the description here into two stages. The first stage was conducted by the business school and the second by the university.<\/p>\n<p class=\"para\">In the first stage, the business school ran an investigation to determine whether it believed Francesca had engaged in academic misconduct. That investigation began on October 27 2021 and concluded almost 17 months later, in March of 2023. In June, 2023 Francesca was put on unpaid leave by the business school after the school concluded that it believed that she indeed had committed academic misconduct.<\/p>\n<p class=\"para\">That began stage two. After alerting her of this possibility in July, 2023, in October, then President Claudine Gay initiated a process to determine whether Francesca\u2019s tenure should be revoked. That process would be conducted by a faculty committee, seven members, what I\u2019m going to call here the Hearing Committee. And for the next year, the business school and Francesca\u2019s lawyers fought about procedures and discovery requests to determine the facts that would be presented to that Hearing Committee. In November 2024, the Hearing Committee conducted a two-day hearing. Two months later in January, it concluded she was guilty. In May 2025 after giving her a chance to appeal that decision, an appeal that I actually wrote, Francesca\u2019s tenure was formally revoked by the university.<\/p>\n<p class=\"para\">So it\u2019s just about four years from the time Data Colada notified Harvard of its allegations until Harvard did something that it had never done in its 385year history: revoke the tenure of an existing Harvard faculty member.<\/p>\n<p class=\"para\">OK. Every problem in this case began with the business school\u2019s special and new procedure for determining academic misconduct.<\/p>\n<p class=\"para\">And it is from this flaw that everything else flows. Consider just two core problems with this new procedure.<\/p>\n<p class=\"para\">First. When you read the description of this new procedure, it\u2019s clear that the people crafting it had been inspired by the procedures that are adopted when someone is charged with sexual harassment or sexual misconduct. In that context, the procedure requires total confidentiality. The target is not allowed to discuss the fact that he or she has been charged. There are good reasons for that. If someone has been charged with sexual harassment or sexual assault and begins to discuss it with other people, then the person who has presumptively been harassed or assaulted could suffer retaliation. It\u2019s important to protect the victim of harassment or sexual assault, and in order to do that, the person charged with harassment or assault is properly told they must remain silent.<\/p>\n<p class=\"para\">But that procedure makes no sense in the context of a proceeding to determine whether someone has committed academic fraud. Because the person who is alleged to have committed academic fraud is the only person whose privacy is at stake. If they wish to discuss the charge or investigate the charges, risking others will learn that they have been charged with academic misconduct, that\u2019s on them. But in this case, Francesca was told she could not speak to anyone. For almost two years, she lived under that gag order. And that gag order, I say, is the first flaw in the procedure.<\/p>\n<p class=\"para\">The second flaw was its restricting Francesca from getting the help she needed to defend herself during this investigation. Because during these 16 months, the business school\u2019s investigation committee launched a process to investigate what it determined actually happened. The committee retained a forensic data analyst to help it interpret the data and to determine whether that data showed Francesca had engaged in fraud. Yet Francesca was forbidden from hiring her own forensic data analyst. She was told she was allowed just two people to support her in her defense, and she had already named two people to that position. So while the business school would have expert advice interpreting complicated data files to determine whether there was sufficient evidence of misconduct, Francesca would have no expert support to help her defend against the charge that she had engaged in fraud.<\/p>\n<p class=\"para\">Now, this restriction is just astonishingly absurd. Why limit the defense at all? What purpose does that serve? How does the business school know that two is the appropriate number of people to help her defend herself in the context of a complicated charge of academic misconduct? And what is the range of skill that those two people are supposed to have? She was advised to get a lawyer and to have a kind of support person, and she followed that advice. No one told her that if she got a support person, she wouldn\u2019t be allowed to hire a forensic analyst. Indeed, no one told her that Harvard itself was going to hire a forensic analyst before she had chosen her two.<\/p>\n<p class=\"para\">To be fair, after the fact, the business school denied that it imposed this restriction upon her. I guess that\u2019s progress, because it at least shows they get why it is so absurd. But Francesca was absolutely clear that she was forbidden. And certainly having raised the possibility herself, had she not been forbidden, you can be sure she would have hired someone. She didn\u2019t because she was not permitted. that is literally what the policy says, and that is what she says the business school staff told her.<\/p>\n<p class=\"para\">Now you might say, why didn\u2019t she just ignore that restriction? Why didn\u2019t she hire whomever she wanted? Why didn\u2019t she talk to whoever she wanted. What were they going to do? Well, what they threatened to do was to terminate her immediately. She had no choice but to follow these rules if she wanted to keep her job while she defended herself. It\u2019s not clear to me exactly how they would have had the authority to terminate her, given the procedure that they had launched was not actually the business school\u2019s procedure. But that was their threat, and it was effective in bending Francesca to their will. She stayed silent during the whole of the process that the business school launched to determine whether, in fact, she was guilty.<\/p>\n<p class=\"para\">Okay, so flaw two then is this: there should have been no restriction on her getting whatever support she needed to defend herself. The idea that she would be excluded from hiring a data forensic analyst when the business school was hiring a data forensic analyst is absurd. The idea that the Investigation Committee would review the evidence the business school had gathered without Francesca having an expert opportunity to review that evidence and rebut it is absurd. The whole process is absurd.<\/p>\n<p class=\"para\">And the bottom line then is that this process that the business school launched to determine whether Francesca was guilty was a catastrophic blunder. It\u2019s almost a caricature of the sort of process a business school would invent. As if the question was, how do you most efficiently convict someone, rather than, how do you most effectively determine whether someone is guilty? There is no equivalent in any proper legal context. You can\u2019t arrest somebody, lock them in a cell, have the prosecutor develop a prosecution against them, but then deny them any real opportunity to build their own defense. Yet that is exactly what the business school did here.<\/p>\n<p class=\"para\">Now the reason I\u2019m calling this a catastrophic blunder is not some abstract theory about due process. This isn\u2019t some highbrow legal analysis criticizing insufficient process, as if process was something we pursue for processes sake. No, the reason this catastrophic blunder is a catastrophic blunder is that we know for certain that the conclusions the process reached were, in critical ways, completely wrong.<\/p>\n<p class=\"para\">Consider just two. As you\u2019ll hear later in this summary, one of the core charges against Francesca, one of the charges that began this whole process, was that she had made up 20 participants in one of her studies. Data Colada pointed to these 20 anomalous entries and suggested she must have added them. The business school committee agreed. But once the business school investigation was over and she was declared guilty, Francesca had her own data analyst look at the data. And once he did, it was conclusively determined that those 20 anomalous entries had been created by a scammer, someone who was simply trying to steal the Amazon gift cards that participants got for participating in the study.<\/p>\n<p class=\"para\">The business school hadn\u2019t discovered this. Their forensic experts hadn\u2019t discovered this. Data Colada hadn\u2019t discovered this. Instead, the business school simply assumed that what Data Colada had alleged, that these 20 entries had been created by Francesca, was true, and therefore that that was evidence of her fraud. And that conclusion grounded their conclusion that she was guilty.<\/p>\n<p class=\"para\">Yet now the Business School and Harvard and I assume Data Colada all concede that this was bullshit. Everybody now understands that these 20 anomalous entries were not created by Francesca, they were created by a scammer. That fact points to this point about process. If the defense had had a chance to examine the evidence before guilt was determined, they would have revealed this error. If the process was as the basics of 101 due process would require, that the defense have a chance to examine the evidence and challenge it, she would not have been found guilty on this charge.<\/p>\n<p class=\"para\">This is the first and clearest example of why process matters and why this catastrophic blunder of due process rigged the procedure against Francesca.<\/p>\n<p class=\"para\">The second way in which we know that this initial flawed investigation produced conclusions we now know are false is even more extraordinary. As I said, to help them build their case against Francesca, the business school retained a forensic expert. That expert produced an over 180-page report that summarized and gathered the data, the evidence that the prosecution believed demonstrated Francesca was guilty. Again, she hadn\u2019t had a chance, prior to the business school concluding that she was guilty, to have that evidence reviewed by her own forensic expert. They simply took that evidence as true, as if, because it\u2019s by an expert that Harvard surely paid an extraordinary amount of money to produce, the conclusions of that report must be valid.<\/p>\n<p class=\"para\">Yet, as we\u2019re going to hear more about later in this episode, after the business school concluded she was guilty based on this report and then proceeded to file charges to have her tenure removed based on this report, and after Francesca\u2019s lawyers spent literally millions of dollars defending her against the charges made in this report, Harvard simply abandoned the report. Francesca\u2019s experts had produced evidence about a bunch of flaws in the report. Those arguments were obviously powerful arguments, because they led Harvard to withdraw the report. Harvard didn\u2019t even believe in the report that had been the foundation to her being found guilty by the business school. Instead, late in the process, it commissioned a whole new forensic report that was released just three months before the hearing was to begin. Now three months might sound like enough time, but the details really matter here. Francesca\u2019s lawyers had submitted a response to the tenure revocation process, the Third Statute Complaint on August 1 2024 and Francesca\u2019s experts were to submit their reports on September 13. By the time the new expert report showed up, Francesca and her experts had less than 30 days to respond. When you think about experts reading other experts reports, then writing their own reports, and then the lawyers trying to understand those reports and finally explain it in briefs, 30 days is no time at all.<\/p>\n<p class=\"para\">No fair process can presume evidence is true or valid. No fair process can introduce evidence of someone\u2019s guilt and not give them a fair opportunity to rebut it. But here, it\u2019s clear that a fair opportunity to rebut the report from the original proceeding would either have required that the report be redone or withdrawn. The faculty members on the business school investigative committee said they had relied upon this withdrawn report in concluding that Francesca was guilty. They could not have relied upon it, had she had the opportunity to show the committee its flaws before the committee concluded she was guilty. Had she had that chance, I cannot believe this committee could have reached the conclusions it did. Had she been given the most basic due process in the first stage of this process, the outcome I am certain would have been different.<\/p>\n<p class=\"para\">&lt; break &gt; So this new procedure initiated by the dean so that he could say to Data Colada that he was responding as quickly as he could to their charges is the foundational catastrophic blunder in the story of this case. It is the mistake from which everything else follows.<\/p>\n<p class=\"para\">But it\u2019s not the only blunder. And the second is just as big.<\/p>\n<p class=\"para\">It\u2019s a basic principle of due process that except for some extreme crimes, there\u2019s a limit to how old a charge can be. Not because we want to give somebody the right to get away with a crime that they\u2019ve gotten away with with a very long time, but because there\u2019s no fair way to reconstruct the record to be able to determine fairly or correctly or accurately whether, in fact, someone\u2019s guilty of a crime.<\/p>\n<p class=\"para\">If the IRS called you and said, &#8220;hey, 20 years ago, you reported an expense as a deduction on your 1040 and we believe that was fraudulent, and we are going to charge you with tax fraud,&#8221; that would be an outrageously unfair charge. Not because you\u2019re necessarily innocent, but because no one, 20 years after filing their taxes, has the evidence necessary to prove their innocence. Actually, I do, because I\u2019m obsessive about keeping records, but most sane people don\u2019t. And that\u2019s why there\u2019s such a thing as what we call in the law &#8220;statutes of limitations,&#8221; statutes that say after a certain amount of time a charge is time barred. Again, not because the guilty must go free, but because even the innocent can\u2019t show their innocence after the evidence is gone. As Justice Jackson put it,<\/p>\n<p class=\"para\">&#8220;Statutes of limitations are statutes of repose. They protect parties from the burden of defending claims after the evidence has been lost, memories have faded and witnesses have disappeared.&#8221;<\/p>\n<p class=\"para\">That principle exists in the context of charges of academic fraud as well. Harvard has expressly embraced that principle. The particular formulation that Harvard adopted was developed by the United States Department of Health and Human Services Office of Research Integrity. It states:<\/p>\n<p class=\"para\">&#8220;An allegation about research that is more than six years old cannot be investigated unless the scholar has continued or renewed an incident of alleged research misconduct through the citation, republication, or other use for the potential benefit of the respondent of the research record in question.&#8221;<\/p>\n<p class=\"para\">That\u2019s a very strong principle, with a really important exception. The principle is that no charge of academic fraud relating to work published more than six years before may be investigated. You cannot be forced to defend yourself about charges more than six years old because they\u2019re too old. But the exact scope of the exception is critically important. You can\u2019t be forced to defend yourself about work that\u2019s more than six years old unless the target of the investigation had<\/p>\n<p class=\"para\">&#8220;continued or renewed an incident of alleged research misconduct through the citation, republication or other use for the potential benefit of the respondent of the research record in question.&#8221;<\/p>\n<p class=\"para\">Now, no one denies that three of the four charges here are more than six years old. Indeed, the oldest of the charges is 14 years old, and the other two, 12 and 11. Thus, under the plain reading of that rule, those charges should not have been investigated unless the exception applies.<\/p>\n<p class=\"para\">Yet they were investigated, and Francesca was forced to bear the costs, literally millions of dollars in legal fees defending herself against these charges, while Harvard has spent, I\u2019m sure, 10 times that amount prosecuting her for these three charges. So how?, or Why? Well, at first, the business school never even defended the idea that they were investigating these three charges, charges that were plainly beyond the six year limit. They didn\u2019t say anything about it. They seemed not even to notice it or consider it. And when Harvard finally explained why they thought they could investigate these charges despite the limitation, why they thought the exception applied, they argued only that Gino had cited the old papers within the last six years, and thereby had &#8220;used&#8221; the old research, and they said that was enough to put them in scope for reexamination under the business school\u2019s policy.<\/p>\n<p class=\"para\">But that claim obscures exactly how Francesca had &#8220;used&#8221; this earlier work. Because all she did was cite the papers. She had listed them on her website. She had cited them from other papers in ways that were not significant, that didn\u2019t point directly to the particular parts that were problematic or alleged to be problematic. For example, within the list of citations in the literature review of the section of a paper these earlier papers were cited. And so the essence of Harvard\u2019s argument is that if you even list a paper on your website or cite it in another paper, that\u2019s effectively using the research in a way that exposes you to further prosecution and investigation for that research. And that, of course, means that there\u2019s no paper that\u2019s exempt from investigation, making the rule, the statute of limitations rule, an absurdity.<\/p>\n<p class=\"para\">Now, among lawyers, this should be just a slam dunk argument. Indeed, the Provost of the University is a famous textualist, one of the leading theorists of textualism in the law. I can\u2019t believe that he would have reviewed this argument and concluded that it was appropriate to prosecute these three charges, given the way Francesca had actually used the earlier research. It is an obviously absurd interpretation of the rule to believe that merely citing an article exposes you to further prosecution for it.<\/p>\n<p class=\"para\">And indeed, the organization that originally crafted the rule has recognized this absurdity or ambiguity and modified the rule to make it clear that it does not allow for the prosecution of a paper merely because it has been cited. As the rule now states, for the exception to apply, the scholar must cite to the portions of the research record alleged to have been fabricated. A simple citation of the paper is not enough. And under this clarification of the rule, Francesca\u2019s citations would not satisfy the exception. Nonetheless, all the way to the very end, Harvard has defended the idea that they can prosecute all four charges when it is absolutely clear, or should be clear, that under the rule, as any reasonable interpreter must interpret it, threefourths of this prosecution was time barred.<\/p>\n<p class=\"para\">Now again, for nonlawyers, I don\u2019t want to be misunderstood here. I don\u2019t think Francesca is guilty of any of the charges made against her. Certainly on the evidence that\u2019s presented, you couldn\u2019t possibly conclude that she was guilty with clear and convincing evidence. And thus to say that the prosecution should not have happened is not to say that she could be guilty. It is to say that it is simply wrong to allow the prosecution to have occurred, independent of whether it should have concluded that she was guilty or not. The protection of the rule is not against being found guilty. The protection of the rule is against being prosecuted.<\/p>\n<p class=\"para\">And I would say to any academic at Harvard who has done empirical work, this part of the Francesca prosecution should concern you the most. Because are you absolutely confident that the research assistants that you use to produce the data that you then analyze to produce your research didn\u2019t make mistakes? Mistakes that might suggest that you juice the data? Because if you\u2019re not absolutely confident, then here\u2019s a bit of free advice: don\u2019t cite the work anymore. Remove it from your website. Because someday there might be a vindictive prosecutor who comes after you, too.<\/p>\n<p class=\"para\">&lt; break &gt; Okay, all that\u2019s the process stuff, the stuff that we lawyers tend to obsess about. Normal people, most people, reasonable people, are just focused on the bottom line, not the process stuff: is she guilty, or is she not? Did she fabricate her data and commit academic fraud? Or is she actually innocent? Are these anomalies, anomalies that were produced unintentionally or accidentally by research assistants in the process of preparing her data?<\/p>\n<p class=\"para\">In the balance of this episode, I will address that question by addressing each of the four allegations in a way to give you an easy, well, not easy, but easy, relative to reading the whole record yourself, way to understand why with each the prosecution for that allegation fails. This is just one episode summarizing at least four hours of episodes. So, of necessity, my summary is abbreviated, maybe not abbreviated enough, because this is a long episode. But if you want to understand the charge in detail, or the charge with any allegation in detail, go back to the original episode which covered that charge. I will refer you to them as we go through this. But it should be enough, this summary, to get you to see the essence of what failed with each of the four.<\/p>\n<p class=\"para\">But if you look across all four of the allegations, there are two consistent flaws that I want to call out up front, because they\u2019ll repeat in each of the four allegations. But they\u2019re significant when seen together. Indeed, they are significant in relation to the point I\u2019ve just made about the burden of proof. Because these two flaws together show, I think, absolutely, why it would not have been possible to conclude with clear and convincing evidence that Francesca was guilty.<\/p>\n<p class=\"para\">The first and most important flaw repeatedly made by the Hearing Committee, the committee that removed her tenure or recommended to remove her tenure in each of the four allegations, is the claim that the data anomalies that had been discovered all tended to support or strengthen the conclusion in Francesca\u2019s papers.<\/p>\n<p class=\"para\">That\u2019s an extremely important assertion, because it seems to provide motive. Everyone concedes there are anomalies in this data.<\/p>\n<p class=\"para\">If they all support the conclusions of the papers, it would be reasonable to believe that they were introduced by someone who had an interest in supporting or strengthening the conclusions of the paper, and that person would be Francesca.<\/p>\n<p class=\"para\">But with each of the four allegations, that claim is just false. Some of the anomalies supported the conclusions of the paper, some of them did not. And that destroys the presumption about motive, absolutely just destroys it. Indeed, rather than support the presumption that Francesca is responsible for these anomalies, the fact that the anomalies don\u2019t all support the paper instead supports the presumption that these are errors. As you\u2019ll see in each case, I describe how these errors could have been produced, like what process or procedure could have produced them. But even without a clear understanding of how they could have been produced, the conflicting character of the anomalies negates the presumption they were intentional and intentionally inserted by Francesca. That\u2019s the first mistake that is common across these four allegations. The second mistake is the mistake of an inadequate investigation.<\/p>\n<p class=\"para\">Because remember something important about the nature of the work that scholars such as Francesca engage in, or people in her field. In a certain way, these scholars run a kind of academic paper factory. Across the period Francesca was working and the papers she was working on that are the subject of this prosecution, she employed, in total, more than 60 research assistants. The job of those research assistants was to collect and clean the data that Francesca and her coauthors would then analyze. They were the ones working with the data.<\/p>\n<p class=\"para\">Now that doesn\u2019t mean they are, of necessity, responsible for any anomaly. But it does mean that an investigation that\u2019s trying to figure out the source of an anomaly certainly must consider the possible role of the research assistants, especially when you can\u2019t say that all of the anomalies are in the direction of the hypothesis of the paper. Across all four allegations where more than at least 12 research assistants were involved Harvard interviewed just two research assistants.<\/p>\n<p class=\"para\">Just think about what that means.<\/p>\n<p class=\"para\">There are errors in the data, anomalies. Everyone concedes that. Conceptually, we know they could have been produced by the research assistants, or they could have been produced by Francesca. But the investigation just presumed it was not the research assistants, so it didn\u2019t investigate with 10 of the 12 research assistants involved, those research assistants, it didn\u2019t interview them. It just concluded, it must have been Francesca who was responsible for the anomalies.<\/p>\n<p class=\"para\">That\u2019s not investigation. That\u2019s presumption. However sensible that presumption would be if indeed all of the anomalies supported the hypothesis of the paper, in a context where the anomalies don\u2019t all support the hypothesis of a paper, that failure to investigate is fatal to the conclusions of the Hearing Committee.<\/p>\n<p class=\"para\">And indeed, I think if you put these two common facts together, there\u2019s really nothing more that you have to think about in the context of this determination by the Hearing Committee. When you put together the fact that, in fact, the anomalies do not all support the conclusions of the papers, and that there was no effort to actually determine who between Francesca and the research assistants could have been responsible for the anomalies, these two facts together absolutely negate the ability to reach a conclusion of guilt with clear and convincing evidence.<\/p>\n<p class=\"para\">I would think even if you had motive absolutely established, you still would need to do an investigation that included one of the two possible sources of the anomalies. But here you don\u2019t have even unambiguous motive. Here you have ambiguous anomalies, plus a failure to investigate. Those two facts together, should have been enough for someone in this process to step back and conclude that no fair process could convict Francesca of academic misconduct.<\/p>\n<p class=\"para\">&lt; break &gt; So let\u2019s consider, then, each of the four allegations and the problems that are present with each. With each of these allegations, I\u2019ve used notebook LM to create cheat sheets that might help you visualize the particular errors. You can see the cheat sheets at the website, theginocase.info as well as interrogate the NotebookLM about the case, ask it questions that my really tediously long summaries don\u2019t provide the answers to.<\/p>\n<p class=\"para\">This is just an audio podcast, so I\u2019m not going to refer directly to that cheat sheet as we go through this. I will instead go through the allegations in chronological order, going from the oldest to the most current.<\/p>\n<p class=\"para\">You can call up the cheat sheet for each of them, if that would help you follow what I\u2019m going to say. So that means we\u2019re going to start with allegation number four.<\/p>\n<p class=\"para\">Allegation number four involves an almost 16-year-old study that Francesca and her coauthors had conducted at the University of North Carolina, which is where Francesca was teaching at the time. The study was completed just as she was moving from North Carolina to Harvard. And the aim of the study was to measure whether the act of signing an honesty pledge would affect the participants truthfulness when reporting their performance on math puzzles. The hypothesis was, if you sign an honesty pledge, that triggers in you a stronger commitment to being honest. And the objective of the study was to measure whether that was in fact true.<\/p>\n<p class=\"para\">Now, 16 years ago, studies like these were not conducted on computers, they were conducted on paper. And what we don\u2019t have now are copies of all of the surveys that were submitted by the participants in the study. Instead, we have a file that was sent to Francesca. I\u2019m going to call that file File A which the Hearing Committee assumed was a complete representation of all the data as compiled by the research assistants and then given to Francesca to analyze. They assumed it was the complete representation of the raw data. And the essence of the charge of fraud is that File A is different from File B, the file she used in the analysis that she ultimately produced, and that difference, the committee said, evidenced the fraud.<\/p>\n<p class=\"para\">But the crux of the problem with this conclusion is the presumption that File A represented the raw data as transcribed from the paper surveys. The Hearing Committee believed File A was the original source file. They assumed, in other words, that this was the baseline file. And they pointed to the differences between the baseline file and the file used to evaluate the ultimate paper, and that difference, they said, showed fraud. So everything hangs on the assumption that File A was the complete representation of the raw data. Everything hangs on that being the first stage in this process, because if it was not the raw data, if it was a file that was in process of collecting the raw data, for example, then there\u2019s no foundation for saying that File B is a modification of the raw data. If instead, File A is just an interim file along the way between the paper surveys and a final data file representing the paper surveys, there would be no basis, in the law we would say no foundation, for anyone saying anyone had modified anything here. And here there\u2019s a plot twist that could have been crafted by Hollywood screenwriters. Because it just turns out that Francesca had the receipts.<\/p>\n<p class=\"para\">As I said at the time, Francesca was working on this project. she was also preparing to move from North Carolina to Harvard. So a moving company packed up all her boxes in North Carolina and moved them to Harvard, and many of those boxes remained untouched in Harvard\u2019s archive. Then when she was removed as a professor from Harvard, another moving company moved those boxes from Harvard to her home. To her garage.<\/p>\n<p class=\"para\">Then when she was preparing to defend herself, in this case, she went through those boxes. And to her extraordinary surprise, she discovered that she, in fact, had the paper receipts for the payments made to the participants in the study. The receipts included participants paid and other studies conducted around the same time, so there are more participants than these in this study at issue. But what these receipts demonstrate was that not all payment records matched with the information in File A, but the payment records did match with the information in File B.<\/p>\n<p class=\"para\">File A was missing participants, which establishes absolutely that File A was not the complete original file, and that instead, File B is the best evidence we have of the completed file. And that means File B doesn\u2019t have anomalies within it. File B is the best evidence we have of what the original surveys were.<\/p>\n<p class=\"para\">Now you can imagine how this would be played if it were an episode of Matlock or Reasonable Doubt. And when I read about the discovery of these receipts establishing that File A wasn\u2019t actually the representation of the raw data, I thought this charge would just be dropped. The receipts proved File A was not a complete representation of the raw surveys. So there\u2019s no foundation to find, at least with clear and convincing evidence, that the data was manipulated or modified by anyone.<\/p>\n<p class=\"para\">But the committee didn\u2019t drop the charge, and it didn\u2019t conclude she wasn\u2019t guilty of the charge. The committee apparently didn\u2019t even understand that the evidence established that File A was not a complete representation of the original data. Francesca\u2019s lawyers had said there were receipts. They testified there were receipts that showed that File A wasn\u2019t complete. They had shown the committee the receipts. But in its final report, the committee stated simply:<\/p>\n<p class=\"para\">&#8220;She did not, however, provide those receipts.&#8221;<\/p>\n<p class=\"para\">This I said in Episode Six was just astonishing. Because at RX 626A and RX 626b in the record, there are copies of the receipts. The committee apparently did not know their own record. Now I don\u2019t blame them. Again, this record was more than 2500 pages long, and the committee was filled with people who have day jobs, and so I don\u2019t know the lawyer who was working with the committee and how extensive their commitments were, and I know that Mistakes can be made. But the point is, once you correct this mistake and realize that the best evidence we have of the original data is File B, there is no way you can conclude that Francesca committed academic fraud.<\/p>\n<p class=\"para\">There is certainly no way you can conclude it with clear and convincing evidence. And I mean that modality quite precisely. When you know File A is not a complete representation of the responses from these participants, that it was at most a working file, that was being modified as the responses were being incorporated and checked, you cannot conclude with clear and convincing evidence that File B is evidence of any fraud.<\/p>\n<p class=\"para\">And then this charge got even crazier. As I described much more extensively in Episode Six, essentially, the Hearing Committee believed that after the experiment was finished, Francesca redescribed the experiment from the one that was actually conducted to strengthen the design of the experiment, like she discovered a mistake in the original design, and so just made up a description of the actual experiment to fix that flaw in the original design.<\/p>\n<p class=\"para\">That sounds bad until you realize that the description of the original design of the experiment describes a completely braindead, logically impossible experiment to conduct. Francesca testified that what actually happened was that the original description was just a mistake, and they corrected the description in the final presentation of the work.<\/p>\n<p class=\"para\">And it was a mistake, they said, because as originally described, the experiment could not work, or at least if causation is forward rather than backwards, it could not work. Well, Harvard responded that it wasn\u2019t just one time that that description was made. It was made twice in two separate documents, which reinforced, in their view, that the original description was the description of what actually happened. One of these documents was submitted to the IRB for approval from North Carolina University, which does make it sound persuasive, until you realize that it\u2019s absolutely clear that the second document is just a copy paste from the first because the very same typo exists in both copies. So this is one document, one document describing an experiment, a brain-dead experiment that is logically impossible to have proved what it was trying to prove, and that description, Francesca said, was just a mistake.<\/p>\n<p class=\"para\">So, there are two theories about what happened here: Number one, a theory that suggests they simply mis described the experiments in that single draft, but conducted an experiment that was eventually described in the published paper accurately. Or, number two, that these supersmart academics planned out a logically impossible experiment and then when it didn\u2019t work, surprise, surprise described the experiment in a logically plausible and sensible way.<\/p>\n<p class=\"para\">When you think about those two theories about what could have happened here, no fair fact finder, I believe, could find, with clear and convincing evidence that the second happened over the first. And that conclusion is buttressed by one way in which this allegation is different from most of the others.<\/p>\n<p class=\"para\">As I said at the start, one common theme throughout these allegations is the failure to investigate completely by failing to talk to the research assistants or others that might have been working with the data.<\/p>\n<p class=\"para\">But in this case, they did question the lab manager, and her testimony negated the idea that the experiment was designed in that completely braindead way, surprise, surprise. She would have remembered if it had been that problem. She said she did not.<\/p>\n<p class=\"para\">But more than that, and confirming the outrageous incompleteness of the investigation, the single most important question to have been asked of the lab manager is the question from the earlier part in this description of this allegation, whether File A was actually a complete representation of the participants in the study. But alas, that question was not asked.<\/p>\n<p class=\"para\">So with respect to the allegation that she improperly redescribed the experiment, the actual testimony that in this case negates that assumption, and, logically, it\u2019s completely absurd to imagine they initially intended and performed the experiment, because that brain dead experiment could not have been conducted. And so there\u2019s no way, with clear and convincing evidence, to conclude that she redescribed the experiment improperly. Instead, she corrected the error in the original description. And with respect to the alleged modifications of File A represented in File B, the evidence shows File A was not the original file.<\/p>\n<p class=\"para\">So we discussed allegation four in Episode Six of the podcast. The cheat sheet on the website nicely summarizes these problems in the allegation. But let\u2019s remember this is the allegation where the paper receipts proved there was no foundation for claiming anything had been modified. That\u2019s allegation number four.<\/p>\n<p class=\"para\">&lt; break &gt; Allegation number three, covered in Episode Seven centers on a 2014 paper entitled &#8220;Evil Genius: how dishonesty can lead to greater creativity.&#8221; The paper included five different experiments, and the one at issue in this allegation was conducted in 2012. So again, we\u2019re talking about events that happened 14 years ago.<\/p>\n<p class=\"para\">This allegation, more than perhaps any other, evinces the consequence of the failures to investigate properly or completely. Because here, as with allegation two and one, there was no effort to question the research assistants who would have worked with these data. But as we\u2019ll see in a minute here, even the most basic questions to those research assistants could have helped determine which of two very different hypotheses was true. These two hypotheses are the obvious ones. Either A, that in gathering and processing the data, the research assistants made these obvious or plausible mistakes, or hypothesis B, that Francesca had manipulated the data.<\/p>\n<p class=\"para\">No doubt the committee and the business school had doubts about whether the research assistants would have remembered what they did with this particular project or any project. Again, that\u2019s why statutes of limitations are such a good idea, just to hark on that point again. But the point is that the questions that would have been asked in this allegation would have been the simplest questions for the research assistants to remember: did you create the data file used in the analysis? Other questions would have been helpful, like, what was your process as you handled data for this study, or which steps were you involved in? It would have been a set of questions, the answers to which would have been remembered, if indeed they remembered anything. And by getting those questions answered, we could have determined whether the thesis that Francesca had manipulated the data here had any plausible basis in fact. Talking to the research assistants seemed particularly key here given that they actually worked on the data for this paper on and off for more than 500 days.<\/p>\n<p class=\"para\">Okay. But let\u2019s start with the specifics of the allegation. The objective this study was to determine whether cheating was associated with greater creativity. The basic design was to create conditions where the participants could cheat and then measure the creativity of the cheaters versus the creativity of the people who didn\u2019t cheat.<\/p>\n<p class=\"para\">Now measuring creativity, of course, involves a subjective judgment. Everyone concedes that. But the way this allegation was considered, everyone, Francesca\u2019s team, Harvard and the Hearing Committee seemed to have believed that determining whether someone was a cheater was an objective judgment, or at least a straightforward judgment. That assumption is understandable because the very first determination the study made was whether someone had lied about guessing a coin flip correctly.<\/p>\n<p class=\"para\">The study asked participants to guess the results of a coin flip. Under Harvard\u2019s assumptions about how the study was conducted, that coin flip was rigged. So if someone guessed heads, the report was tails. If someone guessed tails, the report was heads. So, in fact, everyone guessed the coin flip incorrectly, or at least thought they had guessed it incorrectly. And so, the very first question they were asked was whether they had guessed it correctly. If they said they had guessed it correctly, they were lying. And so, the first variable recorded was whether they had lied or not, liars or cheaters. And so from this perspective, it was easy to say that it was an objective determination whether someone was a cheater or not.<\/p>\n<p class=\"para\">But there\u2019s something really puzzling about the spreadsheet that collected the data that suggests that there was something more to the question whether someone was actually a cheater. Because the raw data for this study has not been found, but the earliest version of the file that the record contains already includes coding by a research assistant processing data in a way to make the analysis possible. And that coding is pretty weird or strange, if you assume that the only question that they were evaluating was whether someone lied or not. If they said they had guessed it correctly, which, of course, was a lie, the column was marked with a one, and that one meant they had lied. If they said they had guessed it incorrectly, that was the truth, and the column was marked with a zero, meaning they had not lied. But the very next column, right next to that column, was a column marked \u2018cheated.\u2019 And in the first available version of the file, the baseline file that the committee used to evaluate whether there had been improper modifications, the values for that column are produced by a simple formula. The formula said: If the previous column, the column that reported whether someone said that they had guessed it correctly or not was zero, then the value in the cheated column is zero, meaning they didn\u2019t cheat. And if the value of the \u2018reported guessed correctly\u2019 column was one, meaning they had lied about guessing it correctly, then in the \u2018cheated\u2019 column, the formula said the value should be one as well. Meaning the \u2018cheated\u2019 column, initially in the first version of this file, produced exactly the same results as the column that said, \u2018reported guessed correctly.\u2019 Or more simply, the \u2018cheated\u2019 column reported the same results as the lying column. And the puzzle this creates is this: Why would the researcher design a spreadsheet with two columns \u2018reported guessed correctly\u2019 and \u2018cheated,\u2019 if they were intended to report exactly the same data? If cheating means lying about the coin flip, the second column is just redundant. Why do you need it if all you\u2019re reporting is whether they had lied about the coin flip?<\/p>\n<p class=\"para\">However, if the study defined cheating in a more complicated way, a multidimensional way, based on other factors in their reporting of the respondents\u2019 behavior, the presence of two columns would be essential, because it would allow a researcher to record that while a participant may have been honest about the coin flip, they still should be considered a cheater because they had violated other rules on the survey. This distinction is the key to understanding a critical part of this allegation, because the allegation is that with 12 participants, the original file reported them as not lying and therefore not cheating. But in the final version of the file, these 12 are reported as not lying but having cheated. So since the original 12 participants who told the truth about the coin flip were eventually marked as cheaters, the Hearing Committee interpreted that change between the two file versions as fraudulent manipulation of the data, manipulated in order to strengthen the study\u2019s finding. But the critical point here is this: That conclusion is only plausible if you assume the two columns were meant to be identical.<\/p>\n<p class=\"para\">And why would you have two identical columns in the data? But if you think the two columns were meant to report two different variables, then the difference between the two columns wouldn\u2019t even suggest that anybody had improperly changed the values. Instead, it would indicate that somebody had properly determined that these 12 were cheaters, even though they hadn\u2019t lied about guessing the coin flip correctly. Okay, so how could someone be a cheater, except by lying about whether they had guessed the coin flip correctly?<\/p>\n<p class=\"para\">Turns out, lots of ways. For example, the survey instructed the participants not to consult information beyond what they knew. That means, if they had gone to the web and looked up answers to the survey questions, that would be cheating. You might say, &#8220;What answers could you look up as you\u2019re trying to fill out a psychological survey designed to measure creativity.&#8221; But it turns out, one of the measures of creativity in this survey was a pretty common measure of creativity. Lots of psychological studies used the same task to measure creativity. So if you wanted to cheat in answering these questions, you could have opened another web browser or tab and done a Google search on this type of question, discovered any number of examples providing answers to these questions on the web, and then just copied those answers into the survey. And if you did something like that, that would be cheating. And if we imagine these two columns were reporting two different things, column one, reporting whether you lied, and column two, beyond lying, reporting whether you did something else to indicate that you were a cheater, then we can understand why the first version of this file could have marked these 12 as not cheaters, but the subsequent processing of the file could have revealed them as cheaters.<\/p>\n<p class=\"para\">Okay. But why, you might ask, would anybody want to cheat? It\u2019s just a psychological survey. I mean, it\u2019s not going to mean you don\u2019t get into law school. So what\u2019s the interest in cheating? What do you gain?<\/p>\n<p class=\"para\">Well, it turns out that the people taking this survey were hired on a platform that Amazon runs called Mechanical Turk. And those people, people who work on Mechanical Turk, have an interest in having a good reputation as being reliable or informed or just good participants in a survey. So those participants would have an interest in getting good grades, because that would help them get hired on other jobs. And thus, they very much would have an interest in getting the right answer or getting as good an answer as they could. They would have an interest, therefore, in cheating. If these two columns were meant to measure two different things, the first column measuring whether they lied about guessing the coin flip correctly, and the second, adding to that judgment other evidence of cheating, then these participants certainly would have had an incentive to cheat, even if they hadn\u2019t lied.<\/p>\n<p class=\"para\">Okay, all that\u2019s a little bit abstract. Let\u2019s get very specific. So after guessing the outcome of a coin flip, the participants in this study were supposed to do two tasks designed to measure their creativity, these two subjective tasks.<\/p>\n<p class=\"para\">One task is called the remote association test, or let\u2019s call it the RAT task. The way that worked was that participants were given three words and then they were supposed to report on a word that was associated with those three words. So if the three words were something like \u2018blank, white and line,\u2019 then the associated word would be \u2018paper.\u2019 The RAT task is a common task in psychological studies, and so it\u2019s easy to find many examples of the word associations on the web.<\/p>\n<p class=\"para\">The second creativity task was called the Usage Task, and here, the participants were given a word and asked to list all the possible uses of the thing the word identified in the real world. So if the word was say, newspaper, somebody could answer the question by \u2018how do you use a newspaper?\u2019 by saying you read a newspaper, it\u2019s a perfectly fine answer. It\u2019s just not very creative. But if you said you could use the newspaper to make a paper airplane or wrap fish or as a substitute for wallpaper, those are also possible uses of a newspaper, but they\u2019re much more creative. And the objective was to measure creativity based on how creative someone was in identifying the uses for the thing named by the word.<\/p>\n<p class=\"para\">But while you could look up the answers to the RAT task on the web, with this task, you really couldn\u2019t look up answers on the web. You couldn\u2019t find answers to the usage task on the web. So if you\u2019re trying to cheat, you could cheat with the RAT task. But cheating wouldn\u2019t change your ability to do well with the usage task.<\/p>\n<p class=\"para\">Okay, here\u2019s where it gets really interesting. Because what\u2019s interesting about these 12 anomalous entries, these entries of people who didn\u2019t lie but were marked as cheaters in the final version of the file, even though in the initial version they were reported as both not lying and not cheating, the interesting thing about these 12 anomalous entries is that they were all really good at answering the RAT task, but just average in answering the usage task. That meant for the sort of thing they could have cheated on, their grades were very good. But for the sort of thing they couldn\u2019t cheat on, their grades were just average.<\/p>\n<p class=\"para\">Now you might say, well, &#8220;If they\u2019re cheating, doesn\u2019t it take longer for them to perform the work than the others who are not cheating?&#8221; And the answer is yes, it would take longer. And indeed, for these 12, the average time to complete their survey was longer, statistically significantly longer than the average time that it took the others to perform the survey. So again, a pattern consistent with a cheater.<\/p>\n<p class=\"para\">Now in the full episode, I go through some of the other indicators, all of them suggesting that these 12 were cheaters. But the only important point to see here is when you look at the actual answers they give across a range of questions, it would have been perfectly reasonable for a research assistant to conclude that these 12, though they hadn\u2019t lied about guessing the coin flip correctly, could still have been properly considered cheaters.<\/p>\n<p class=\"para\">Now, again, we don\u2019t know exactly what the research assistant who was preparing the data here was told to do or actually did, but here is another clear example of where even a simple investigation could have disambiguated between two very different hypotheses.<\/p>\n<p class=\"para\">Hypothesis number one that the research assistant was doing exactly what I said. An investigator could have asked the research assistant, &#8220;Do you remember what you were doing with respect to this column marked \u2018cheated\u2019?&#8221; If indeed they were instructed to make a holistic determination of whether the participants had cheated, they would remember that. They would say that. Then we would have absolute evidence that these changes were not anomalies produced by Francesca to strengthen the conclusions of her paper. We would have absolute evidence that they were instead the research assistant doing the work they were hired to do, preparing the data to be analyzed.<\/p>\n<p class=\"para\">But if the research assistant had said &#8220;No, all we did was to code the column marked \u2018cheated\u2019 to be the same as the previous column,&#8221; then you would have pretty good evidence that Francesca had modified the data, or somebody had in order to strengthen the conclusions. So the point is obvious: A simple investigation could have resolved this completely.<\/p>\n<p class=\"para\">But even without that obvious question being asked, the point is that the strong support for why these 12 could have been marked as cheating, even though they did not lie, means that there\u2019s no way you could conclude with clear and convincing evidence that they\u2019re being marked as cheaters means Francesca had modified the data to strengthen the conclusions of her paper.<\/p>\n<p class=\"para\">Okay. Those weren\u2019t the only problems that the Hearing Committee found in the data in allegation number three. I go through the other problems in tedious detail in Episode Seven. But I think we can summarize that tedious explication quite simply. Yes, we could identify anomalies in the data, and for some of those anomalies, we have the evidence of how those anomalies were produced.<\/p>\n<p class=\"para\">For example, with the second subjective task, the Usage task, we can show that the research assistant copied data from the wrong location in a spreadsheet and that produced the error. But with this error, we can be confident it was a mistake made by the research assistants. Shouldn\u2019t be any ambiguity of that. And why can we be confident that it was a mistake made by the research assistants? How do we know that copying from the wrong location was unintentional rather than intentional? Because the result of that copying was to weaken the conclusions of the paper. Francesca had no reason to modify data to weaken her study.<\/p>\n<p class=\"para\">So with respect to that error, we can say it\u2019s error because there\u2019s no reason to believe Francesca had intentionally copied this data that weakened her study. Now, not all of the mistaken copied data weakened the study. Some of the mistakes strengthen the paper. But you know, that\u2019s the nature of errors: they go both ways.<\/p>\n<p class=\"para\">And the critical point is this, having established that there were errors weakening the paper, that destroys the inference that these differences were intentional differences designed to strengthen the outcome of the paper, because they were the same sort of errors made in the same context.<\/p>\n<p class=\"para\">Now, as I noted in the episode, it\u2019s not possible to explain all of the errors for the participants. One in particular, the RAT scores for four participants, didn\u2019t add up each of the answers in each of the 17 questions for four of them, they just didn\u2019t add up. Francesca can\u2019t account for that error, neither can I see how that might have occurred.<\/p>\n<p class=\"para\">But with respect to this problem, and with respect to an allegation that there was some effort to cover up problems with the data when the data was shared with one of the members of the Data Colada team, the critical point again, is this: Any uncertainty here could have been resolved by talking to the research assistants. A simple interview could have distinguished between the scenario where it was plausible that the research assistants made the mistakes and the scenario where it seems clear that Francesca must have been the person responsible for the mistakes or the errors, and if Francesca then clear it would have been data fraud.<\/p>\n<p class=\"para\">Okay, drawing the allegation together. It is clear that some of the anomalies in this allegation were just errors caused by research assistants, because they were plainly weakening the conclusions of the paper. And I said that fact sweeps in the other errors as well, even some that strengthen the paper, because they were all errors of the same sort. Their source is similar, and it is likely to be the same sort of error. With the 12 participants, if indeed the two columns were to represent two different variables, the changes aren\u2019t errors. They\u2019re not anomalies. They are the research assistant doing his or her job. The evidence shows why these participants could have been marked as cheaters, even though they were not liars. The only way to prove with clear and convincing evidence the contrary assumption, the assumption that this was fraud, was to interview the research assistants. Yet again, something Harvard couldn\u2019t be bothered to do.<\/p>\n<p class=\"para\">The bottom line for this allegation is this: the profound weakness in this allegation is caused by the failure to do the most basic investigation by interviewing the research assistants who had worked with the data. That weakness destroys the ability to conclude with clear and convincing evidence that Francesca had modified his data.<\/p>\n<p class=\"para\">That\u2019s allegation three. Let\u2019s take a break.<\/p>\n<p class=\"para\">&lt; break &gt; Okay, now we get to allegation number two, covered in Episodes Four and a brief Episode Five. I\u2019ve already described a little bit of the scandal around this allegation at the start of this episode. This is the allegation involving the 20 scammer entries. So allegation two involved a paper titled &#8220;The Moral Virtue of Authenticity,&#8221; and a study conducted in 2014 that was reported in that paper. I described the paper in episode four, but that detail of what the paper tried to do is not important here.<\/p>\n<p class=\"para\">Instead, what\u2019s important is the allegation of fraud. And The core of this allegation is what I\u2019ve already described. This was the claim that there were 20 fraudulent entries created in the survey which the business school assumed Francesca must have entered herself, even though what made them seem anomalous was the error in answering the most obvious questions that anybody knows who knows anything about university life would have answered correctly. The question was, &#8220;what year were you in school?&#8221; And rather than listing first year or freshmen, the answers were Harvard. Now as a professor at Harvard and as a person who spent time on college campuses with college students, that\u2019s not the sort of error Francesca would have made, and that seems pretty obvious to me. But apparently it wasn\u2019t obvious to Data Colada or to the business school or the business school\u2019s data expert.<\/p>\n<p class=\"para\">They looked at these anomalous entries and they concluded Francesca must have entered them. Now as I described at the beginning of this episode, when Francesca was able to hire the experts who could analyze the data using the metadata from Qualtrics, those experts used the IT forensics to demonstrate absolutely these 20 entries were scammers that had nothing to do with Francesca. And ultimately, the Hearing Committee agreed this was not Francesca.<\/p>\n<p class=\"para\">She was innocent of this charge. Then in addition, there was a claim by the business school that there was a gap between the data in the study and the data it was published. Data that was published to something called the Open Science Framework, suggesting that Francesca had made up the observations that were in the published version because they were helpful to her conclusion. And that charge also helped convince the Business school Investigation Committee that Francesca was a fraud. She\u2019d made up these data. But nine months later, the expert who had made those allegations admitted that they were completely wrong and that there were no missing entries.<\/p>\n<p class=\"para\">They failed to make that notice transparent to the Hearing Committee, but they admitted that this charge was wrong. So that\u2019s two for two. And you might have thought that having recognized that the basis for the charge had been removed once Francesca had had the ability to review the forensic evidence, you might have thought the business school would have withdrawn the charge at that point. But no, that\u2019s not what happened.<\/p>\n<p class=\"para\">Instead, the business school hired a new expert. And the new expert went through the same data. And the new expert agreed that the 20 anomalous entries came from a scammer. But they then focused on 154 altered cells, anomalous data, and 154 cells. And between the source data and the final file, those 154 entries, this new expert suggested, suggested that Francesca had engaged in academic fraud.<\/p>\n<p class=\"para\">Okay. Now here again, the Hearing Committee made its false claim that all of the changes strengthened the paper\u2019s hypothesis, thereby suggesting motive. But in reality, only about 80 of the 154 changes strengthened the results. The rest of the changes affected variables that were totally irrelevant to the hypothesis of the paper.<\/p>\n<p class=\"para\">But still, there was the question. Still there is the question: how did these changes come about?<\/p>\n<p class=\"para\">And here the point of episode four was quite important. The theory of the Hearing Committee concluding that Francesca had engaged in fraud by altering these 154 cells to strengthen the paper, even though 44% of the 154 did not strengthen her argument at all. The assumption of the Hearing Committee was that she would have moved through the cells and made the changes to strengthen the results that would be produced by the data.<\/p>\n<p class=\"para\">But Francesca\u2019s experts discovered a pattern in the changes, a pattern that would be explained by the RA working on the data, making a very particular kind of Excel error. when manipulating the data. That error is replicable. We can take the original file, make that error and produce the resulting file exactly. And so the question I pressed in Episode Four was this: given two alternatives, alternative A, that an Excel error explains the 154 differences, and alternative B, that Francesca moving through the file randomly made these 154 changes or differences that were observed, some benefiting her, some not. Which of the two is more likely?<\/p>\n<p class=\"para\">And it wasn\u2019t just these two points that I was trying to get you to think about, which of these two is more likely. Instead, in episode four, I tried to make a sharper point. What I said was, what are the chances that B would produce alterations with the same structure, the exact same alterations as were reported from assumption A? Or again, when you have one way to account for the changes, through a replicable error made in manipulating the Excel file, what are the chances that a random set of changes would replicate those changes precisely?<\/p>\n<p class=\"para\">Or again, or put more strongly: There are two hypotheses here, an innocent hypothesis and a guilty hypothesis. The innocent hypothesis is that this Excel error produced the changes. The guilty hypothesis is that these changes were the result of intentional manipulation. But, it just so happens that the random but intentional manipulation produced the same 154 changes as the innocent hypothesis does. When putting these together, my point was simple: What are the chances that the guilty hypothesis is true, given it produced exactly the same results as the innocent hypothesis? If Francesca wanted to strengthen the results by manipulating the data, she had 2455 observations in that spreadsheet to select among. 2455 cells that she could have changed. So what are the chances that she would have happened to select the same 154 changes that the innocent explanation identifies? I mean, that\u2019s a probability question, and the math to do that probability question is way above my pay grade. But even as a lawyer, I can tell you the chances are obviously incredibly small. A person randomly editing cells would edit them, yes, randomly. But these cells have a pattern in rectangles more consistent with the kind of drag errors for which Excel is, for better or worse, notorious.<\/p>\n<p class=\"para\">Okay. Finally, the committee was convinced that these changes had been made by Francesca, because they were all made on a day when RAs were unlikely to be working, Thanksgiving 2014, 12 years ago.<\/p>\n<p class=\"para\">But as Episode Five demonstrated, very short episode, but one, I thought I had to add, because someone raised this question to me, you can only reach that conclusion, the conclusion that it all happened on one day by willfully ignoring the evidence provided by Francesca\u2019s lawyers and experts that there were at least five earlier files in the archives leading up to the file worked on on Thanksgiving 2014. Because, in fact, the file that was worked on on that day in 2014 was a file that had been worked on over many months. Only by ignoring these other inconvenient files that can the committee assert that she must have made those changes in a single 24-hour holiday window.<\/p>\n<p class=\"para\">That\u2019s allegation two. And to summarize, and use your cheat sheet, if it\u2019s going to help you see what I\u2019m summarizing here, I\u2019m going to call that strike three. Strike one was the bogus claim about the 20 participants. All concede that was the work of scammers, not Francesca. Strike two was the allegedly made-up data. Oops, Harvard\u2019s first expert confessed no data was actually made-up. And now strike three is this, 154 alterations, 44% of which have nothing to do with the conclusions of the paper, all explained by a plausible Excel error with an effectively 0% chance that the same pattern of changes would have been produced by simply tweaking 3% of the potentially conclusion strengthening variables. Let\u2019s call this allegation the scammer allegation. This allegation two, like four and three fails.<\/p>\n<p class=\"para\">Okay. Finally, allegation one. Now notice again, because I won\u2019t let you forget this. This is the only allegation made within the six-year period of the rules. The only allegation that should have been investigated under Harvard\u2019s own rules. The only allegation that Francesca should have had to defend against.<\/p>\n<p class=\"para\">The paper at issue in this allegation was published in the Journal of Personality and Social Psychology in 2020. It was titled, &#8220;Why Connect? The Moral Consequences of Networking with a Promotion or Prevention Focus.&#8221; Again, the details of the paper aren\u2019t important, though we explain them in Episode Eight, in some detail. What is important is the nature of the fraud charge.<\/p>\n<p class=\"para\">The charge of fraud involved 1066 anomalous entries that the Hearing Committee concluded Francesca must have made, because they said they all strengthened the study, and so they all evinced motive. Okay, but surprise, surprise and again, as with each of the other allegations, that is simply not true, 415 of the 1066 anomalies, 38% had nothing to do with the hypothesis of the paper. So once again, this was not a case of clear motive. It was a case of mixed evidence, which destroys the presumption of motive. Almost 40% of the changes had no impact on the study\u2019s hypothesis. On the assumption Francesca was making these changes, why would she have gone through the trouble of making these irrelevant changes.<\/p>\n<p class=\"para\">But what got the Hearing Committee going here was that they thought they had a slam dunk case demonstrating Francesca had made the 1066 alterations. And the evidence for their slam dunk case was this. That on the day Francesca shared the written-up results with her coauthors, earlier in that day, she had downloaded the raw data for the file used in the analysis of those results from Qualtrics. And because she had accessed the raw data on that day, but then shared results based on a file that differed from that raw data, differed because of these 1066 alterations, they concluded she must have been the one making those 1066 alterations. And then, even more deviously, they alleged she had spent most of the day<\/p>\n<p class=\"para\">&#8220;using SPSS running commands in a manner consistent with repeatedly altering the data and then checking whether it improved the results.&#8221;<\/p>\n<p class=\"para\">Okay, that\u2019s an important quote. Let me say it again. They said she had spent the day,<\/p>\n<p class=\"para\">&#8220;using SPSS running commands in a manner consistent with repeatedly altering the data and then checking whether it improved the results.&#8221;<\/p>\n<p class=\"para\">Now that\u2019s a pretty strong charge. And someone making that charge is implying that they looked at the only evidence that could support that charge, the log for the SPSS session, and had drawn from that evidence conclusive evidence to support that claim. Or again, the Hearing Committee is saying that the SPSS logs support the claim that Francesca basically spent the day obsessively tweaking the data to juice the results of her study.<\/p>\n<p class=\"para\">Now not to give away the punch line here, but that claim is utter bullshit. But to see just how utter that bullshit is, we need to unpack a little bit more what the Hearing Committee claimed.<\/p>\n<p class=\"para\">The first bit of weirdness in the Hearing Committee\u2019s claim is the suggestion that Francesca had done something with this data that she never did. That she had started from the raw data set and cleaned the data to prepare it to be analyzed herself. As I\u2019ve said, as I\u2019ve explained throughout the course of these episodes, that work was the work done by research assistants. Yet the Hearing Committee\u2019s claim is that on this one occasion, Francesca demoted herself to the level of a research assistant, and spent the hours it would have taken to prepare the data to be analyzed, and then spent the hours it would have taken to repeatedly modify the data, to juice the results, to produce the paper. Weird.<\/p>\n<p class=\"para\">The second bit of weirdness in the Hearing Committee\u2019s claim is that it is flatly contradicted by Harvard\u2019s own expert. The Hearing Committee asserted that the data for the file that Francesca turned over on January 24 must have been downloaded from Qualtrics on January 24. But Harvard\u2019s own forensic expert testified that he didn\u2019t believe that the source for the file that was shared on January 24 had been downloaded from Qualtrics on January 24. He believed the data file existed before January 24. Inexplicably, the Hearing Committee ignored Harvard\u2019s own forensic expert. And he was not the only one who said that the data file that was worked on on January 24 had been created prior to January 24. One of Francesca\u2019s RAs had testified, and again, this was a rare case, when the RAs had been interviewed, that he had worked on a version of the file before January 24 and he had given her a copy of that file before January 24. The record doesn\u2019t include the file, or didn\u2019t include the file anymore. But it did include a directory that did name a file that was named in a way to indicate that it was provided by this research assistant, Alex Rohe. The file had the initial \u2018R\u2019 added to the name. It was a data file, that sort of file Francesca would have used or taken from the research to use to analyze the data.<\/p>\n<p class=\"para\">But the committee rejected this testimony. Why? Well, because it turns out there was another file on the computer that also had this mysterious letter R in its file name. But when that file was created, Alex Rohe was not a research assistant for Francesca. And so the Hearing Committee deduced that the letter R was not a convention used to refer to the name of the research assistant. QED.<\/p>\n<p class=\"para\">But what the Hearing Committee missed is that at the time this other file was created, there was a different research assistant working for Francesca, Mindy Rock. Notice her initial is also R. And so the second file actually supports the suggestion that there was a convention to name data files with the initial of the RA working on that file.<\/p>\n<p class=\"para\">Nonetheless, the Hearing Committee was not to be discouraged. They were convinced that the data file that Francesca turned over to her co-authors on January 24 had originated as the data she had downloaded from Qualtrics on January 24.<\/p>\n<p class=\"para\">Okay, but the crux of the allegation made against Francesca, the most important part of that allegation is the claim by the Hearing Committee that she spent the afternoon and evening of January 24 working with these data files,<\/p>\n<p class=\"para\">&#8220;using SPSS, running commands in a manner consistent with repeatedly altering the data and then checking whether it improves the results.&#8221;<\/p>\n<p class=\"para\">That\u2019s a pretty damning conclusion. The suggestion was she just sat there, repeatedly running the same commands over and over again, tweaking the data to strengthen the results of the paper. And as I noted in Episode Eight, that sounds pretty bad. But as my collaborator, the Anonymized Ava, observed, it was also complete and utter bullshit.<\/p>\n<p class=\"para\">Because Ava, leveraging her powerful data skills, actually took the log files from those SPSS sessions and examined them. The actual commands were interspersed with about 63,000 lines of code automatically generated by SPSS as Francesca used SPSS menu. But if you strip away the junk, it\u2019s possible to see what Francesca was actually doing on SPSS that afternoon and evening.<\/p>\n<p class=\"para\">And what did Ava discover? It wasn\u2019t that Francesca was running the same commands on the same data set over and over and over, tweaking the data set to juice the results. Instead, what the actual logs showed was that of the 61 distinct commands that Francesca performed that day only six, six out of 61, occurred more than once for the data set at issue. Four of them occurred twice, and two of them occurred three times, none more than that. That means the claim that she spent the whole day perpetually running the same commands over and over and over again to juice the data was utter bullshit. It just did not happen. The objective evidence from the log file shows it just did not happen.<\/p>\n<p class=\"para\">Now here again, some of the 1066 changes that were identified did not strengthen the results of the paper. And again, as I\u2019ve said throughout the episode, where there are changes that both strengthen and don\u2019t strengthen the results of the paper, that should negate the inference of motive. Because there would be no reason to produce changes that don\u2019t strengthen the results of the paper if you are intentionally trying to fabricate results.<\/p>\n<p class=\"para\">So where did the anomalies come from? Well, here again, Francesca offered testimony about how a common kind of Excel error could have produced those anomalies. And again, that evidence was ignored.<\/p>\n<p class=\"para\">But for me, the real kicker with this allegation comes at the very end. Because it\u2019s conceivable we could have known exactly what happened on that computer that day. If Harvard had done its job competently, we could have known exactly from where Francesca got the data that she worked on that day. We could have determined exactly when the file that she worked on was opened and from where it came.<\/p>\n<p class=\"para\">And we could have done that because computers are spies. Built into the operating system of a computer is a persistent surveillance mechanism. That mechanism gathers data about what you do on your computer and records it in computer logs. And that surveillance mechanism reports, for example, or could report that a thumb drive was inserted at 12:41, that it would have reported whatever identifying information there was about that thumb drive. It could have reported that a file was opened from that thumb drive at 12:42, it could have reported that the thumb drive was removed at 1:12. That surveillance mechanism reports that information and stores it in log files. And a forensic expert can read those log files and determine precisely what happened with that computer on that day.<\/p>\n<p class=\"para\">And so, in this case, a forensic expert could have been able to read these log files and determine precisely whether Francesca did as Francesca said she did, or whether Francesca did as Harvard alleged she did. That data, in principle, was available.<\/p>\n<p class=\"para\">But as it turned out, because of the keystone cops investigation that Harvard launched against Francesca, that data was destroyed.<\/p>\n<p class=\"para\">Why was it destroyed?<\/p>\n<p class=\"para\">This is really the most extraordinary part to me. When Harvard notified Francesca she\u2019d been charged with academic misconduct, it seized her computers. They told her that they had taken, &#8220;a forensic copy&#8221; of the &#8220;hard drives&#8221; of the computers.<\/p>\n<p class=\"para\">Now what that means for anybody who knows anything about conducting an investigation involving computers is that they basically take an image of the computer. I don\u2019t mean a photograph. I mean they take an image of the state of everything on the computer at the time the investigation begins. An exact copy of the entire hard drive of the computer. And sometimes even more, but at least the entire hard drive of the computer, so that they\u2019re able to perfectly reconstruct the entire state of the computer at the time the charge of misconduct is made.<\/p>\n<p class=\"para\">It turns out Harvard didn\u2019t do that. Instead, they just sent a technician down to Francesca\u2019s office and had him copy off of the computer certain files they thought would be relevant to their investigation. Which means all of the log files were vulnerable to being overwritten depending on when the computer was fired up again, because the log files have an automatic routine that deletes them after a certain amount of time.<\/p>\n<p class=\"para\">So When Francesca was finally able to hire a forensic expert, and that forensic expert was able to do the work necessary to defend Francesca, they made a request to Harvard to produce the copy of the hard drive. The forensic copy of the hard drive. Harvard reported they didn\u2019t have one. They didn\u2019t have one because all they had were copies of files taken from the computer. And by this time, the log files were all gone.<\/p>\n<p class=\"para\">The files that could have shown, absolutely, Francesca\u2019s innocence were gone because of the inept investigation Harvard had launched against Francesca.<\/p>\n<p class=\"para\">Now Ava commented at this part of Episode Eight that she understood and felt sorry for the technician. What did he know about computer forensics? But Harvard\u2019s a big institution. The business school is an especially rich institution. And the idea that they would run a third-rate investigation of the data forensics necessary to establish whether a tenured member of their faculty had committed academic misconduct is itself misconduct. The simplest evidence Francesca could have pointed to to objectively establish her innocence, oops, Harvard destroyed through its own incompetence.<\/p>\n<p class=\"para\">Okay, that\u2019s allegation one.<\/p>\n<p class=\"para\">The committee fabricated a scenario that Francesca was doing something Francesca never did: preparing her own data. Despite Harvard\u2019s own expert testifying, that\u2019s not what happened; despite Francesca\u2019s testifying, that\u2019s not what happened; despite Francesca\u2019s RA saying he had prepared the data files in advance of this one magical day, so that\u2019s not what happened; despite the fact that the objective evidence from the SPSS data log shows absolutely without doubt that the scenario of manipulation that the Hearing Committee concluded Francesca had engaged did not happen.<\/p>\n<p class=\"para\">Nonetheless, the committee concluded that there was, &#8220;clear and convincing&#8221; evidence that she had done what there was literally no direct evidence she had done. And then, to add insult to injury, the incompetence of Harvard\u2019s own technology staff meant she had no opportunity to use simple direct evidence to prove what wasn\u2019t her burden to prove, but to prove her innocence.<\/p>\n<p class=\"para\">Let\u2019s call this keystone cops allegation number one. And that\u2019s the last allegation against Francesca. Let\u2019s take a break.<\/p>\n<p class=\"para\">&lt; break &gt; In late 2024, the Atlantic Magazine published an extraordinary piece titled &#8220;The Fraudulent Science of Success.&#8221; The piece began with an account of Francesca\u2019s case and relied heavily on people like Nick Brown and James Heathers, who were in the business of analyzing data sets, looking for anomalies. The piece then pivoted to talk about Juliana Schroeder, who was one of Gino\u2019s coauthors. And after Francesca\u2019s story broke, Schroeder began to audit their joint work. They launched a project called the Many CoAuthors Project, which gathered all the joint work and tried to isolate errors in the work. Indeed, the project discovered many errors. None of the errors it discovered were tied to Francesca. Critically, the piece then pivots to Schroeder\u2019s own research, and it reveals that in Schroeder\u2019s own research, there were problems that were uncovered during the self audit. The author of The Atlantic piece presses Schroeder to account for these errors. And after confessing that she was, &#8220;deeply ashamed&#8221; of the errors discovered, the article describes her response as follows.<\/p>\n<p class=\"para\">Still, she said that the source of the error wasn\u2019t her. Her research assistants on the project may have caused the problem. Schroeder wonders if they got confused. She said the two RAs both undergraduates, had recruited the women at the gym, and that the scene there was chaotic, sometimes multiple people coming up on them at once, and the undergrads may have had to make some changes on the fly, adjusting which participants were being put into which group for the study. Many things went wrong from there, Schroeder said. One or both RAs may have gotten ruffled as they tried to paper over inconsistencies in their recordkeeping. They both knew what the experiment was meant to show and how the data ought to look. So it\u2019s possible that they peaked a little at the data and reassigned the numbers in a way that seemed correct.<\/p>\n<p class=\"para\">But the Atlantic author wasn\u2019t satisfied with that account, and two pages later, he continues,<\/p>\n<p class=\"para\">I spoke with Schroeder for the last time on the day before Halloween. She was notably composed when I confronted her with the possibility that she had engaged in data tampering herself. She repeated what she told me many months before that she definitely did not go in and change the numbers in her study, and she rejected the idea that her self-audits had been strategic, that she had used them to draw attention away from her own wrongdoings.<\/p>\n<p class=\"para\">And then in the final pages of the essay, the author struggles with the ambiguity and with the increasing recognition of many in the field, that the field is marred with many similar problems, many examples of data just not adding up.<\/p>\n<p class=\"para\">But what\u2019s striking to me when I read stories like this is how vigorously people hold on to the least likely explanation for the problems they\u2019ve identified. As I\u2019ve said throughout these podcasts, this is a weird field of science where the work is almost factory produced, where there are many people involved in the production of the science, and where, frankly, not much actually hangs on the results. This is not like Boeing screwing around with safety standards. No plane is going to fall out of the sky if there\u2019s an error copying data in an excel sheet. So, it\u2019s not hard for me to understand how people would not take seriously enough the need for precision in the work that they do as they prepare these data. Especially when you understand who\u2019s preparing the data, undergraduate research assistants, people with a million other things going on, people for whom absolute accuracy may not seem as important as answering that next snap.<\/p>\n<p class=\"para\">I don\u2019t say any of this to excuse data fraud. I don\u2019t say any of this to suggest that data fraud doesn\u2019t exist. But I do say it to suggest that anyone raising a question about the work of someone else in this field, or I\u2019d say any field, needs to start with the obvious and only with compelling evidence move to the unlikely. They need to start with the presumption that this is error and only with the most compelling evidence move to the conclusion that this is data fraud. Or, as the Atlantic author put it, data tampering.<\/p>\n<p class=\"para\">Because between error and evil, the vast majority of cases will be error, and in that world, the real evil is attributing to others an intent which you don\u2019t have the evidence to support. That\u2019s what I believe happened in this case.<\/p>\n<p class=\"para\">It was totally appropriate for Dana Colada to raise questions. Those questions should have triggered a careful investigation by the Harvard Business School. They didn\u2019t. They triggered an embarrassment. A plain denial of fair process, denying Francesca an opportunity to understand, to explain or to defend the basis for the charges against her. And then once the bureaucracy got going defending its decisions, the machine could not stop. Until it did stop, with the judgment from the Hearing Committee that ignored the failure to investigate the most plausible alternative accounts for the anomalies that they reflected upon and who repeatedly asserted what was plainly false, that all of the anomalies strengthened the conclusions of the paper.<\/p>\n<p class=\"para\">Of all the simple ways to understand what went wrong here, here is the simplest. When you have an allegation of data manipulation or data fraud, and in fact, the anomalies are ambiguous, as they were in this case, that should end the investigation until there is clear evidence that the target of the investigation did the manipulation. Evidence, such as the evidence that was lost with Allegation One when the university failed to preserve a forensic image of the computer, the investigation should end.<\/p>\n<p class=\"para\">Now you might say, &#8220;Well, what if an evil genius recognizes that\u2019s the rule? What if they create changes that don\u2019t strengthen the paper, while they also create changes that do strengthen the paper?&#8221; I agree, that\u2019s a real problem. And I agree we need to take special steps to investigate and prosecute that problem. But no one alleges in this case that Francesca was that sort of evil genius, and even if they did, there is no evidence in this case to show, with clear and convincing evidence, that that\u2019s what she was, or that\u2019s what she did. The discipline that this process demanded was for someone to ask cleareyed and directly, could the evidence here meet the standard? No one asked and answered that question.<\/p>\n<p class=\"para\">At some point, serious people in this process must have recognized the embarrassment that was the business School\u2019s process. They did nothing to stop it. At some point, serious people realized that the foundation for the judgment made by the investigation committee of the business school had crumbled. They did nothing to stop the process. They just hired new experts to find new reasons to continue the prosecution. At some point, serious people realized the fundamental unfairness in forcing Francesca\u2019s lawyers after spending a year preparing for one case to remake their case within 30 days to defend against a slew of new charges by a new expert, but they did nothing to stop it. And at some point, whether the President of this University or someone in his administration must have looked back at the work that had been done and seen the problems and recognized the injustice, but they too, did nothing to stop it.<\/p>\n<p class=\"para\">And to this day, an army of lawyers defending and rationalizing their work continues that work, they continue to press. Indeed, press even further, as I\u2019ve learned, they\u2019ve now tried to force me to turn over my emails claiming that I can be deposed because I would have evidence to answer the question whether Francesca was guilty.<\/p>\n<p class=\"para\">&#8220;What do you expect?&#8221; a friend said to me. &#8220;You\u2019ve become a traitor.&#8221; Really? A traitor? A traitor to an institution whose brand is &#8220;Veritas&#8221; \u2014 truth.<\/p>\n<p class=\"para\">What I see in truth is four allegations raised against Francesca found to be true by a Hearing Committee, but that no fair application of the standard that that Hearing Committee was to apply clear and convincing evidence could find those allegations were true. And to the extent I\u2019ve succeeded in raising in your head questions about any of them, we should recognize that there is a question only because Harvard failed to conduct a proper and adequate investigation.<\/p>\n<p class=\"para\">I\u2019m here today, this case is here today because its target, Francesca Gino, refused to shut up and go away. When the business school concluded she was guilty, they tried to get her to quietly resign, as they and the university have done with many other professors in the history of Harvard who turned out to embarrass Harvard because of their misbehavior.<\/p>\n<p class=\"para\">But Francesca refused, because Francesca did not commit academic misconduct. She fought back. She fought back with all of the resources she and her family had. Even then, that wasn\u2019t enough, and it required the help of others, including the help of Bill Ackman, who, after careful study of these allegations, concluded, as I did, that the allegations are false.<\/p>\n<p class=\"para\">So where should Harvard go from here?<\/p>\n<p class=\"para\">Well, first: I\u2019m not under any illusion that a podcast is going to remake the world. I\u2019m pretty sure that Harvard will continue to ignore what we\u2019ve said here, what the story reveals, and just get on with the many other fights that it has, some of which I think are the most noble fights in America today. There\u2019s litigation happening now. I\u2019m not sure what will happen in that litigation. I\u2019m not sure what could happen in that litigation.<\/p>\n<p class=\"para\">But I still think Harvard should step up and do the right thing. At a minimum, the President of Harvard should say it was wrong to investigate and prosecute Francesca for at least three of the four charges. And if they want to continue to insist on the correctness of the prosecution of the fourth charge, allegation number one, then they should have that allegation evaluated by an independent fact finder, an actual fact finder, an actual person who understands how to evaluate evidence against a standard. Maybe a former judge. Indeed, there are former judges on the Harvard Law School faculty. They could listen to the evidence and make an evaluation of whether there is any possibility of finding with clear and convincing evidence that for the one allegation, allegation number one, Francesca Gino had committed academic misconduct.<\/p>\n<p class=\"para\">And when that process concludes that there is absolutely no chance that a properly applied, clear and convincing standard could have concluded that Francesca is guilty, then the university needs the courage to admit it, to admit they got this one wrong and to correct the mistake they created.<\/p>\n<p class=\"para\">God knows the error was understandable. I wrote the appeal to the Hearing Committee\u2019s judgment, and we filed it just at the moment the University was under its most extreme pressure from President Trump and his administration. I know they were distracted. I know a lot was going on. I\u2019m not saying anybody was malicious or evil.<\/p>\n<p class=\"para\">They were just not courageous. They were just not committed to Veritas. The bureaucracy got going, its inertia was unstoppable. No one was going to have the courage to stand up and say, this is not right. It was easier for everyone to just go along, easier for everyone to just let it happen.<\/p>\n<p class=\"para\">When I was a student in college, I spent a summer in Eastern Europe and the Soviet Union. It was 1982. On a very hot day, I was in a bus crossing between Romania and Bulgaria. The line at the customs checkpoint was endless. I think it took us an hour to get across that border. I spent the time looking out the window. This was 1982, there was no TikTok or Instagram, but the images out the window were extraordinary. Nonetheless, it was Eastern Europe in the 1980s. The cars were not BMWs, yet they were beautiful and elaborately filled with all sorts of things. Boxes tied to car roofs. Trunks not quite closed, but fastened to stay shut.<\/p>\n<p class=\"para\">A couple cars ahead of us, there was a truck. The truck was carrying cages of geese. At one point, the truck hit a bump, and the gates of the truck opened, and a bunch of the cages fell onto the road. Geese were everywhere, just as many racing into the traffic as off the road. But rather than slowing the slow roll of cars and trucks, the rolling continued. Truck after car after bus just rolled over the geese. They squealed as they were crushed. But no one could be bothered to stop to clear the way to fix the problem.<\/p>\n<p class=\"para\">I\u2019ve thought about that scene many times across my life. I\u2019ve thought about it here as I\u2019ve tried to help Francesca tell her story. I\u2019ve so much wanted to see someone stop, someone have the courage to stop. Someone break the rhythm. Someone have the courage to do what\u2019s right.<\/p>\n<p class=\"para\">But courage doesn\u2019t mark us at this time, in this place, even at this university. And for that, I am very sorry. But that sorrow is but a fraction of the sorrow I have when I think about the injustice that Francesca Gino has suffered. We should be better than this, especially, we should be better than this.<\/p>\n<p class=\"para\">&lt; break &gt; I think this is the last episode of this podcast. It\u2019s the last for now. I am extraordinarily grateful to Ava for her incredible work across the course of this investigation, helping us understand the facts and discovering many of the reasons why we can conclude with confidence that Francesca is innocent. I\u2019m grateful to Francesca for letting me tell this story. I don\u2019t know how anyone goes through what she has gone through, certainly not with the poise and integrity and courage that she has shown. There have been moments of weeping. You\u2019ve heard some of them on this podcast. But I see only strength. Maybe it\u2019s easier to be strong when you know your rights, but maybe it\u2019s harder. Maybe knowing your rights and yet facing what she\u2019s facing is the most depressing reality, I hope, I don\u2019t know.<\/p>\n<p class=\"para\">And finally, I\u2019m grateful to you, listeners. Many of you have written and some of you have offered support. We have no clear ask of you, other than to spread the word to the extent you think there\u2019s word to spread. I hope some will help reform the reality of the process at Harvard and if not just at Harvard elsewhere too, because some of what I\u2019ve heard is parallel stories about this kind of problem everywhere across the world. We can\u2019t let that happen without fighting to resist it. And if you can help in that elsewhere or here, please do.<\/p>\n<p class=\"para\">Thanks to Josh Elstro of Elstro production for working through these difficult podcasts and the corrections and the edits necessary to make sure that they are presented as professionally as he presents them.<\/p>\n<p class=\"para\">This is Larry Lessig.<\/p>\n<p class=\"para\">Thanks for listening.<\/p>\n<\/div><footer class=\"ep-footer\"><a class=\"ep-footer-link\" href=\"#gino-index\">\u2191 Back to index<\/a><\/footer><\/section>\n\n  <\/div>\n<\/main>\n\n<button type=\"button\" class=\"back-to-top\" id=\"gino-back-to-top\" aria-label=\"Back to top\">\u2191<\/button>\n\n<\/div>\n\n<script>\n  (function () {\n    const btn = document.getElementById('gino-back-to-top');\n    function onScroll() { if (window.scrollY > 480) btn.classList.add('visible'); else btn.classList.remove('visible'); }\n    window.addEventListener('scroll', onScroll, { passive: true });\n    onScroll();\n    btn.addEventListener('click', () => window.scrollTo({ top: 0, behavior: 'smooth' }));\n    window.addEventListener('keydown', (e) => {\n      if (e.target && (e.target.matches('input, textarea, [contenteditable]'))) return;\n      if (e.key === 't' || e.key === 'T') { e.preventDefault(); window.scrollTo({ top: 0, behavior: 'smooth' }); }\n    });\n  })();\n<\/script>\n\n<!-- ===== End Transcripts block ===== -->\n\n","protected":false},"excerpt":{"rendered":"<p>The Law, Such As It Is &middot; Season 3 The Gino Case \u2014 Transcripts Full transcripts of all nine episodes of the third season of the podcast &#8220;The Law, Such As It Is,&#8221; reviewing the tenure revocation of Harvard Business School Professor Francesca Gino. Episodes 01The Story BeginsLarry Lessig introduces the season and the essential &hellip; <a href=\"https:\/\/TheGinoCase.info\/?page_id=162\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Transcripts&#8221;<\/span><\/a><\/p>\n","protected":false},"author":3,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-162","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/TheGinoCase.info\/index.php?rest_route=\/wp\/v2\/pages\/162","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/TheGinoCase.info\/index.php?rest_route=\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/TheGinoCase.info\/index.php?rest_route=\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/TheGinoCase.info\/index.php?rest_route=\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/TheGinoCase.info\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=162"}],"version-history":[{"count":16,"href":"https:\/\/TheGinoCase.info\/index.php?rest_route=\/wp\/v2\/pages\/162\/revisions"}],"predecessor-version":[{"id":181,"href":"https:\/\/TheGinoCase.info\/index.php?rest_route=\/wp\/v2\/pages\/162\/revisions\/181"}],"wp:attachment":[{"href":"https:\/\/TheGinoCase.info\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=162"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}