Latent Space - 🔬Biology Is Turning Into Software — Matt McPartland & Neil Patel, Chai Discovery

Chai Discovery co-founder Matt McPartlon and product lead Neil Patil explain why pharma abruptly started buying AI design tools this year instead of forcing ...

Transcript

0:00 · looks a lot less like a, you know, a chat GBT and a lot more like uh Autodesk or or Solid Works or or Figma, you know, if you've used those things where you can kind of load up your molecule.

0:10 · There's this almost like Photoshopesque like design suite. You have this equivalent of a paint tool to kind of paint your epitope. You have this equivalent of a contentaware fill tool to kind of get your uh your binders generated from Chai. And I think to add to that, right, yeah, this notion of target discovery and hit discovery and optimization where each of these has a gate and takes a few months to a few years is this very like waterfall model, right?

0:32 · Where the cost of trying things and getting things early is very expensive. [music] But I think to what Matt's saying, right, if you start to get in a regime where you can have models give you really promising candidates, you can start to make that look a lot more like a loop, right? It's it's akin to like becoming more agile in software development. But now the next problem is like agonists, right? like how do you reliably oneshot hitting a switch like on a cell, right? Or buy specifics or ADCs, right? And I think this uh levels of abstraction that we're going to have to climb with the product as like the models get better.

1:02 · If you have like these really good primitives for structure prediction and binding and design and you can kind of compose them, then you can start to just like grow into like the outer loop of science.

1:15 · Welcome to Len Space AI for science. I'm Brandon. I build RA therapeutics at A atomic AI. I'm joined by my co-host R.J.

1:22 · Honiki, CTO and co-founder of Mirror OMIX. It's a pleasure to have with us in the studio today Magma Partland and Neil Patil of Chai Discovery. Uh Chai is a protein design startup which is about 2 and a half years old and has made quite a splash in those few years. They have several very exciting announcements that I think they'll tell us about today. But yeah, to get started, could you two give us a bit about your background and your uh what you do at Chai? Yeah, thank you very much for having us. We're super excited to talk about Chai today. Uh I'm Matt McPartland. I'm one of the co-founders of Chai.

1:52 · Uh my background is in like AI biology related stuff during my PhD. Um I actually started my PhD in like theoretical computer science and then transitioned to this later. Yeah, I I've been doing this stuff now for like about 8 years and I kind of came into the field at an interesting time where protein structure prediction was like just starting to see signs of life. So this is like Alpha Fold one days. Um, and was in the field during Alphaold 2 and like got to see a lot of the interesting developments at that time.

2:23 · So yeah, I I'd always been pretty interested in like applying this stuff in the real world and Chai was just a perfect opportunity to do that. And I'm Neil Patiel. I help lead a platform and product here at Chai. So a lot of the stuff around infrastructure to train models, serve them, and then the productization piece, you know, the design suite that lets you use the models. Um, I kind of have a more meandering path. So, I kind of got into programming like 15 years ago, making apps in the app store. Got really addicted to the dopamine hits you get from that. Uh, and then actually got nerd sniped by robotics and like worked on that for a bit. Self-driving cars in like 2018, 2019.

2:55 · Got really jaded and was like, I don't want to touch hardware for a while. Ended up switching and joining a SAS company called Vanta as one of the first employees there and kind of grew with it. Started my own security company afterwards. Got a few years into that and I was like, you know what, Adams are kind of cool. like I want to work on something a little more meaningful and so uh I joined Chai about a year ago um right after Chai 2 uh was announced to help with a lot of the platform and commercialization pieces.

3:21 · Awesome. It's like the five stages of grief or something.

3:24 · Yeah. [laughter] Yeah. We're we're at acceptance.

3:27 · Awesome. Um you have these I think four now big partnerships and uh raised a whole bunch of money. Can you tell us a little bit about those partnerships and then what I really want to know is what are you telling investors and customers that is so compelling that they're willing to do these big deals?

3:45 · Yeah.

3:45 · So, uh like we've been very fortunate to partner uh first with Eli Liy and then with Fizer Noardis and our Gen X. Um yeah, I think it's been like a really interesting ride and I think our business model is also very compelling to a lot of people. Um, like we really like to we care about the partners succeeding like this. Chai as a company really depends on how the partners succeed. I think Neil probably has some interesting takes on like, you know, what we actually offer and what makes that so compelling. So, I'll hand it over to you.

4:13 · Yeah, I mean, as you all know, drug discovery is a very lengthy process, right? And a lot of these pharma companies are spending lots of time, you know, years and years and billions of dollars trying to find initial therapeutic candidates. And so at Chai, you know, we train models that can help accelerate that process and kind of find those initial binders and and then some.

4:31 · And you know, we, you know, there's a lot of bio companies, AI for bio companies that are like making their own drugs. We really don't see ourselves that way, right? We we see ourselves as almost a a neutral software factory for making medicines. And so, um, that's what, you know, lets us go then work with and support all of these other farmers, um, in their kind of drug discovery journey. Um and so yeah, I mean a lot of this capital is just another proof point that um we can sort of uh start to really accelerate that software factory, right?

4:57 · Go after harder modalities, train bigger models and ultimately just build what our our partners and customers ask us for.

5:05 · But what is it that why you and not other structural companies? Why are why are they compelled to buy from you? The thesis of Chai has always been to like be the software and modeling layer which was I think like very controversial at the time like everyone you know this this play is definitely two years ago and it's already like a completely different world. Yes.

5:26 · Yeah.

5:26 · It's pretty crazy like the like people tried this play for a while. Um and I think like the models just really weren't there yet. Uh, and even like for us, we were taking a risk in the very beginning. Like we were kind of banking on the models getting there. And like I had seen early signs of life in my work and our CEO Josh like he he was on the original ESM papers on that team at Meta and he was seeing like pretty early signs of life that like you know there might be scaling laws here. Uh they like I think we'll actually be able to start like designing things. Structure prediction is getting really good.

5:52 · Like one one like crazy thought is like we didn't have a multimeter structure prediction model until like 2021. That was 5 years ago when we could like start with deep learning to like actually predict the shape of two proteins at once. Like it was a alfold one was like and alold 2 was like this huge breakthrough. But then like alfold 2 multimemer came out like a year later.

6:14 · So like you really kind of needed that to unlock design in the first place anyway. Like we weren't even trying to predict multiple proteins at once. Uh and then really like around that time inverse folding kind of started working and was like oh protein mnnn this actually works in the lab. Like credit to the Baker lab for doing all this really excellent lab validation on all their models, but I think like we're starting to see them do interesting things and like actually work on like real world experiments. Uh, and now is probably the time to start betting on this.

6:40 · I think like before then maybe you could take like some experimental data from a campaign on like this one target that you had and you care about and you you might be able to like make some progress on that and like keep hill climbing in this like one very specific case. general models weren't really a thing back then. So, I think like yeah, we took that bet pretty seriously and like we we decided to just like push as hard as possible and to really like shoot for generality in our approach.

7:05 · And then when Chi 2 came out um our second paper after Chai 1, uh we kind of like showed the world like this is actually possible and it's possible at scale. We didn't show this for like one or two targets like it kind of works like we were like let's just go all in.

7:18 · I think uh Josh likes to say we set a bold companywide challenge uh to design antibodies to 50 targets and it actually like we saw some signs of life. We're like all right let's like let's do this with real statistics and see if this actually works. It's an interesting story of how we chose these targets. So we were like all right what targets we going to choose we should choose like some interesting targets whatever. Uh and at that point we were like kind of ramping up with CRO's and figuring out like what what does our wetland process look like?

7:44 · Uh and we decided uh after after trying some stuff with like many proteins whatever we're like here are the interesting targets. This is what we should look at and like half the time the targets just like kind of didn't work. We were still learning whatever and we're like all right maybe we should just go with like targets that the CRO have actually validated. So let's get the CRO catalog see what they've already worked on restrict that to like an interesting set.

8:04 · Uh so from that we chose 50 targets designed antibodies against them. uh got hits to half and at that point I think pharma starts to realize like okay there actually signs of life here and this this might actually work in some of our programs and so antibodies is maybe a more challenging domain than other structural prediction problems. So why tackle antibodies? So maybe back up what is an antibbody?

8:31 · Yeah

8:31 · and what do you do with it that and why is it an attractive target? The analogy that everyone gives like this lock and key kind of problem uh where like your target this protein that you're trying to bind to it might be some like disease protein uh that's kind of like your lock and then you want to design this key that fits into it and like in in our case just like sticks there. The interesting thing with antibodies is like these like really flexible general proteins like in a lot of ways they're very general in a lot of ways they're actually like pretty uniform. Uh but at least like how they bind to a target is very general.

9:01 · So like you have a lot of optionality in how you design this kind of binding interface. Um the structure prediction problem for antibodies like predict how this antibody actually binds to the target, how it how the key fits into the lock. That's been a notoriously difficult problem. Uh the nice thing is like so we we've made a lot of progress on structure prediction. Kind of the field as a whole has come a long way uh along like in in getting structure prediction to where it is.

9:25 · But in the design setting, uh, you can be a lot more selective about the types of designs you want to make and the types of structures you actually want to focus on. And in some cases, it might actually be even easier to design a protein binder that is an antibbody than to actually predict how it might bind that target in general. So like it's kind of like if you if you have the freedom to choose, you can kind of just pick the easy cases if that makes sense.

9:48 · So the antibodies like there's a whole machinery in the body that works with antibodies. what what does the body do with it naturally and what can you do with um that is sort of not natural but is useful for therapeutics. This is coming from from a non-biologist here but I think of antibodies like they're these kind of like Y-shaped proteins. So like kind of looks like a P sign with your fingers. Each each of these uh fingers is kind of like uh an arm of the antibbody.

10:16 · And like it's really actually only the tips of the of your fingers, the tips of the antibody that engage in binding. So this makes these really like nice therapeutic design targets for that particular region uh reason. The nice part is that like the rest apart from the tips is like actually relatively constant. So this is called like the framework region of an antibbody. In the design problem you're typically just designing like the very fingertips and you can actually choose for the most part like these kind of framework regions that your immune system already recognizes.

10:45 · So antibodies kind of like these Y-shaped proteins that your immune system like recognizes. It knows really well. It's kind of like your body's it's one of the lines in defense against pathogens and other types of diseases.

10:58 · So, so I guess uh antibodies can on the one end like connects to proteins on the surface of a cell typically or other things but typically on the surface of a cell and then the other end helps the immune system identify a pathogen typically. But you can also do things like you mentioned ADCs, anti- antibbody drug conjugates. So that's that means putting a drug on the other side or something like that and that causes the when you bind to something that it releases the drug into the cell, right?

11:28 · They're like this very general framework, right? Where kind of on the ends you have these CDR loops and you can design them to kind of bind to arbitrary things where maybe one end you bind to a cancer cell, the other end you bind to a toxic molecule. You're now precision delivering that toxic molecule to a cancer cell, right? or you just have two ends bind to things and kind of force like induced proximity um to have some effect in the body. Or, you know, a lot of drugs historically are really just like about like blocking things, right? Like anti-agonist behavior, right? Um, but maybe you can have agonist behavior.

11:59 · We actually like really precisely like press a switch.

12:03 · Like there's a a GPCR protein which are these like doorbell proteins that sit in your cell membrane. You have an antibbody like very precisely engineered to to poke it in a certain way that causes a downstream chain reaction. And I think like one of the things that's really exciting about where we're getting to with some of these models is we can start to get that precise, right?

12:22 · We can really target a very specific epitope, right? Meaning like binding spot, right? A very specific set of atoms to have the antibbody go after. Um which, you know, historically you're with a lot of drugs, you're just kind of brute forcing, you know, a lot of antibodies and just trying to come up with a bunch of things and see what sticks. But maybe that gets you a binder to some spot of your target molecule, but that doesn't let you precisely engineer where you're poking after.

12:45 · I know you're not biologists, but do you have any of like idea about how they used to design these before you know these models came up? Like what would you what was the grueling process you would do to find or what is which is actually still Yeah.

12:58 · What still is the state-of-the-art in terms of drugs which have made it to the clinic?

13:02 · Yeah.

13:02 · Josh uh our CEO likes to say that our uh our biggest competitor is the mouse uh so like or or nature in in certain ways. So like uh traditionally these these types of like drug like molecules were either discovered in like these immunization campaigns. So like you literally will just like infect a mouse with a disease and see what antibodies it makes to try to like combat that. Um other ways of doing this is like super large yeast display so on.

13:27 · And so you might like start with, hey, I really like this framework and how am I going to like figure out the right loops to design to bind this target. I'm just going to try as much as I possibly can and just like literally search for a needle in a hay stack. Uh, and this would be like on the order of like at least billions of potential molecules that you're screening against this one target. Uh, and in that case, you might like end up with, you know, one, two, maybe like a dozen potential hits to this target. You actually, you don't know much about those hits.

13:52 · All you know is that they kind of like stick to the target. you don't know necessarily where or like if they're even necessarily drug-like. I think like one big separator of chai and like a thing that definitely our partners like to see is like you can be really intentional with how you want to do this this design process. You can say I want to bind this target in this particular area. You can even go back and look to the designs after like we validated that our designs. So you can go back and look and say like is this antibody engaging the target in the way that I expect? Do I think this will actually have the therapeutic effect that I'm going after?

14:24 · One of the cool things about knowing that you have the right binding pose is that you can now also design selectivity into that. Does your platform have some technique for doing selectivity? Yeah, there there's a there's a nice mix of uh ideas that went both into the modeling side and especially on the product side for for dealing with selectivity and cross reactivity. Uh so in some cases you want your molecule to bind uh one target and avoid another one.

14:48 · So you might have like healthy variants of protein and like disease variant of protein. you want to avoid this this disease variant or you might have some other similar protein that's like not actually harmful in your body that you don't want to just like artificially block. So I think like on the modeling side, yeah, we've come up with ways of doing that, but I think it's even more interesting on the product side. So like how do you enable customers go through or partners to go through and like actually intentionally design for these things?

15:15 · Yeah.

15:15 · And maybe to like back up and define cross reactivity, right? Like it turns out when [clears throat] you're developing a drug, you're not necessarily going straight to injecting that into a human, right? Like you might want to put it in monkeys first, for example. And the monkey might have a maybe mostly similar but slightly different variant of it. And so your drug, you know, not only needs to bind to the human variant, but also the monkey variant, right?

15:35 · Um and so um you know the way we've tried to model the models and the product is to kind of let you account for those very general cases where you say hey I'm trying to design something that can bind to both of these things so that I can actually go and develop the drug. Let me actually identify maybe the the region that's conserved and then target conserved means you know doesn't change much between the two and target that exact region.

15:57 · And then you know similar with cross uh with selectivity right maybe you might want to there's a very similar protein in the human that if you accidentally bind that one uh that's very bad and you only want to bind the target protein and you know that's why a lot of drugs right you know fail or or toxic or have you know really bad side effects right and so um it's kind of you're you're kind of having this like combinatorial problem of like you know bind only these things and avoid only

16:22 · these and uh I think what what's been really exciting with some of the progress recently has been like a lot the improvements we've been able to make on the level of specificity we get we can get to uh with those models.

16:33 · So you're not only designing the bind here but you're also making sure that the that it doesn't bind to another thing. So that other ways that like carties have tried to tackle this by having some molecular or some sort of signaling pathway that says if I I bind I only fire if I bind this one binds and this one doesn't bind. But you're saying you just divi design a an antibbody that actually only will bind to the thing that you care about.

16:58 · We're getting to the point where in some case I mean it's nuanced, right? But in some cases you can actually try that.

17:02 · Okay.

17:02 · That's amazing. Yeah. So so you you're saying [clears throat] you essentially call it counter screen or you have in part of your platform you can know reliably counter screen against like a large diverse set of proteins which might be issues for downstream.

17:16 · Yeah. I would say the framing is more you can be very specific about what you care about binding versus what you care about avoiding. But I think you know for example like a lot of the money that we're raising now will let us train bigger models that can maybe be even more general and start to account for even more things at the same time.

17:31 · Right.

17:32 · Maybe we should back up. Let's talk about um so the history of the Chai, you know, series of models.

17:39 · Um well, why don't you tell the story?

17:41 · We started Chai around two and a half years ago. At first couple months we're like, "All right, we're we're we're gonna work on protein design." Uh, and we were working on this. We're making some progress. We're like, "Oh, this is pretty interesting." Like, we had some ideas and models. And then kind of like that was right when Alfold 3 came out.

17:57 · And we were we' like been talking about like, man, we really need like an MSA pipeline. We need like all this infrastructure.

18:03 · MSA is multiple sequence alignment.

18:06 · Why is this just We've covered this before, but what is a MSA like in two sentences and why is it important? So if you want to predict the structure of a protein, uh it might be really useful to see a bunch of very similar protein sequences. And what those protein sequences that are really similar tell you is like kind of what positions like which amino acids end up being conserved across many variants of this protein.

18:26 · And if you see like high levels of conservation or like kind of high levels of uh mutation like correlated mutations, it typically gives you some indication that these amino acids are close in 3D space. So you kind of have this like 2D view of a protein which can then be used to help you predict this 3D structure.

18:42 · So you're learning from evolution what was conserved because the things that weren't conserved probably broke the protein and something died or didn't make it.

18:50 · Exactly.

18:50 · Right. Yeah. Yeah. It's it's it's pretty remarkable that this works honestly. One of my favorite like bio facts here. Um yeah. So so we were like kind of thinking like oh man it'd be it'd be nice to have like a lot of infra and whatever. So Alphold 3 came out.

19:03 · We're like hey we should we should like open source this model. we should just like you know bunker down build all the info that we need. Uh I think like this will pay back like in the long term for sure of just like as a forcing function to like be where we are and also just like to contribute to the community as a whole.

19:19 · So it's interesting that you chose okay this we're actually what we're doing here we're building a model but what we're really doing is learning how to build the infrastructure. Is that kind of what you're saying?

19:27 · Yeah

19:27 · that's exactly right. And like I I had built a lot of like similar infrastructure in my PhD but not at a production level for a company. Uh so like at that point I think we were five people. Uh so there were five of us at China and we're like all right this is our forcing function. We have like a clear goal to work towards. It's like very direct. Let's get this thing going and see how fast we can do it.

19:47 · You guys were at this time sitting in the open AI office. Is that we were sitting in the open AI office?

19:51 · Yeah. In the in the mission, right? So like what's the backstory on that is really interesting.

19:55 · Uh two of our other co-founders Josh and Jack had a relationship with some of the uh OpenAI people actually. Uh OpenAI co-led our seed round. So we were like kind of thinking like all right should we get an office while we're only five people and it turned out like that office was mostly vacant. So we we got to sit in on the like in the open a offices for a while. Taiwan built it open source learned about infrastructure.

20:19 · Yeah.

20:19 · So then after that like we we really set the sights down on protein design and worth pointing out chai one was a structure prediction model right. So you have the you have the sequence what is the structure that it folds to and then that was chai 2.

20:32 · Yeah

20:32 · chai one. Try one's finished. Uh, one one other crazy story there. Let's see if we can actually share this. Uh, but this is a hilarious one. So, like we we were like, "Oh man, we really want to be the first to put this out." And we were like, "Okay, we're we're one week out. We're like the model's like almost done training." We're like, "Should we should we build a web server?" And then we're like, "Oh yeah, maybe not." And then like we ended up spinning up like this whole web server so like people could use it like rather than just like download the git repo. It's kind of annoying especially for biologists. And like we actually wanted people to use this. So, like, let's spin up a web server.

21:02 · Uh, let's get the technical report out, all this stuff. So, we we ended up like we were up for like 48 hours straight, just like getting the paper over the line, getting the like all the last things done on the web server, and then Josh was interviewing with uh like Bloomberg TV or something that morning. And we've been up for like 48 hours straight. So, Josh like runs into a room to do this interview on Bloomberg TV. And like uh I think it was like 7:00 in the morning. Everyone's in the office. Like we we didn't like want to be seen, whatever. like the interviewer is like, "Oh, like interesting company.

21:32 · Doesn't look like there are any employees here."

21:36 · [laughter] Uh but yeah, it was it was a really fun time. I think like the early startup days were just just super fun. So yeah, after that we kind of set our sights on design and really what we were thinking is like uh we we kind of always had antibodies in mind. We thought of this as like the most tractable problem. The nice thing with proteins is you you have this beautiful sequence representation.

21:56 · And there's already a lot of research been done in like how do you auto reggressively generate sequences? How do you like this sequence generation problem is well studied. Uh so we were thinking like what what's a nice like area to apply sequence generation to in in the bio space and it it's pretty natural to do like linear sequences of amino acids. So we start working on design. A unique thing about THI is like we're not like we're designing antibodies like we're an antibody company like we don't we don't really like pigeon hole ourselves into like one therapeutic area. So we like tried to really tackle this problem very generally.

22:27 · So we were thinking like can we design many proteins? Can we design antibodies? Can we scaffold regular complexes uh so like really just take a holistic view on like how do you design proteins in general? Uh and that eventually led to the CHI 2 model. So that was our our like first flagship design model and that's where uh the Chi 2 paper uh and like our bold target discovery project came in. So we designed antibodies to 50 targets for that paper. got binders to about half of them with I think on average around a 20% hit rate for binding.

22:57 · Uh and then afterwards started working on CHI 3. So that's our our latest series of model but a break there.

23:06 · But before we talk about Chai 3, can you tell us about especially for listeners that may not be familiar with structure prediction models, what does the model look like? How does it work in general?

23:15 · Let's take a look at Chai 1. Uh, Taiwan has this like roughly a tokenizer, a transformer, something that looks like a language model, and then something that kind of looks like an image diffusion model, and they're all just like stick stitched together. The tokenizer is like not your kind of typical like words of X style tokenizer. Uh, this is like I have a bunch of atoms in a molecule, and now I want to like pull those into what I would call tokens for my like LM looking trunk. uh and then that conditions this like kind of big diffusion model which will then emit the image which is some 3D structure.

23:48 · So is it atoms or is it amino acids that are the input?

23:52 · It's an interesting question as well. Uh so we we have like all these different input tracks. So like one thing about biology is the data is inherently multimodality in a sense. Uh you have these this like you know kind of token sequence representation. Each of these tokens has like a set of atoms that kind of dangles off. And then you also have, you know, some some properties of the different atoms. Like an atom is might have like a different charge. It might have a different element type. So like periodic table of atoms. Um and then these kind of all get bunched together into tokens.

24:22 · Once tokenized, you can kind of process this in very standard ways. Uh but then ultimately you have to get back to these like 3D coordinates. So like in order to predict the structure this is just some 3D object and that object goes through or like to emit that object you go through what looks like an image diffusion model where you kind of go back from from tokens back to the atom representation.

24:44 · I see. So the tokens go in, the transformer establishes the relationship between the different tokens and then the diffusion model turns that represent that latent representation into a 3D structure.

25:01 · That's exactly right. Yeah.

25:02 · Okay, great. So that's CHI 2. That was Tai one.

25:05 · Okay.

25:05 · Okay. So try one folio model.

25:07 · Yeah. It's like uh and like all this bio stuff. sounds like kind of scary to like atoms, tokens, amino acids. Uh like at the end of the day, my background personally is like theoretical computer science. Uh that's what I spent like all of my earlier years doing transition to this like pretty late in my PhD. But I think like the background that you need is really similar to the background that you need for like any other field of machine learning. There are all these domain specific things that you learn about.

25:32 · Uh, but like one one analogy or like anecdote I like to like to say is people think you can't work on like AI bio unless you're a biologist. But it's kind of like you can't work on like video models unless you're like a director or something. Like there are all these like super domain specific things like oh yeah to understand like lighting and a video things like that but at the end of the day these are just machine learning problems and like there they're all solved the same way.

25:55 · Okay. So then chai 2 there's a jump in capability as well as an architectural change right?

26:00 · Yeah.

26:00 · What we've disclosed about try 2 is like it is an all atom diffusion model. Uh so we we're trying to predict like you know atoms in 3D space still but we're doing it in such a way that like the model actually has the ability to like design atoms place them decide which atoms actually are there. So like one way to represent amino an amino acid like a protein token is by like which atoms are present.

26:23 · So in the chai 2 case we were just predicting like all right show the model let the model just kind of pick what atoms it wants to keep uh and then map that back to what amino acids there are. What are you able to do with chai 2 that you can't do with chai one is just like better or is it are there new capabilities it brings it's design right so chai 1 lets you say

26:45 · hey I know the sequence of amino acids right that text string and I know the structure that you would get from the genome or exactly chai 2 says okay I have a target structure right that I want to design a binder to um how try to will then generate you know candidate molecules candidate medicines that bind to that target. Um, and so this is kind of a design model or design family of models.

27:07 · And I think that's where you really cross the threshold of usefulness, right? Like I mean, try one alpha very useful because you can, you know, you can at least intuit it and and reason about the structure and and see what you're looking at. But, you know, the ultimate goal here is to design medicines, right? And design new molecules. And I think try to really cross the threshold of performance for doing that with antibodies a year ago.

27:27 · One analogy here would be like um kind of like back to to like the image domain. Uh so like try one would be like you know there is a cat in this image like thanks try one and try two is [laughter] like I like I'll show you a background maybe like I'll prompt you with some some like image information like hey uh put a cat in a field and try to actually just like give you back an image of a cat in a field and you're like that that's a good-look image or it's not. You might have some other model which kind of ranks the image. Uh but fundamentally it's the generative problem.

27:55 · Yeah.

27:55 · So there's taking that analogy a step further. It's maybe more like you show it a background and then it generates there is a cat and then it generates an image of the cat at the same time and it makes sense that there is a cat in this field and also that the cat works in the image. So there's a it is a it's an interesting problem because you have to generate two things at the same time both the sequence and the structure. Can you if you I don't know if you can but could you talk a bit about like how that works? Like how do you do that? So you code you co-design the sequence in a way that the structure also fits and makes sense.

28:26 · One way to think about it is um kind of like the classic way of doing this. Let's talk about both in structure prediction. Like all right, I know the sequence and like I can from that roughly figure out the 3D shape. Uh and then there's kind of like the inverse folding problem which is like given a 3D shape, give me back a sequence that would fold into this. And now you kind of like need to do both things at the same time. But I think like similar principles apply like you can kind of have the model like think a little bit about what should this structure look like? Then you can have some other part of the model thinking about like now what sequence would maybe support this.

28:57 · And then like a nice thing with diffusion is like you can do this pretty slowly and pretty iteratively. So you can give the model a lot of time to think about all right if I change the structure like this how should the sequence change uh and you can kind of just play this back and forth and back and forth and eventually it ends up kind of converging on something that's self-consistent.

29:14 · It's almost like a EM algorithm.

29:16 · Yeah, exactly. [laughter] So you have this model now chi 2 which is able to predict or to to sample a structure and a sequence which generates that structure and just because you can generate a structure like doesn't necessarily mean it's necessarily accurate enough to do something. So do you have other scaffolding on top of that?

29:37 · Are there additional problems like are you oneshotting these things or are you you know needing to generate thousands of them and then you have a ranking or scoring or you know how like just having a candidate is maybe let's say not enough. Um, so what do you do once you sample a structure or traditionally what's what's done and like when when codeesign and like uh protein structure design like started to become a thing, we're like kind of at a loss for metrics is like how do you know that your protein like you design some some like sequence in structure? Like how do I know that this is legit or not?

30:11 · Like I can tell you it's like anything.

30:12 · It's like totally out of domain now, right? By definition.

30:15 · Yeah.

30:15 · Yeah. And like as a human you can look at this thing and be like I I don't know. It checks out. Like even biologists are like, I have no idea if this thing actually folds. Like maybe some of it looks right. Uh even our biologists are surprised by the way with like some of our designs that like do end up working. What was done at the time is like we kind of came up with a bunch of metrics and like alphafold it really is what enabled this. Um so you'd take the sequence that you predicted.

30:37 · You'd run that through some like totally uh like distinct structure prediction method. So this is completely independent of your model and you say if an independent model thinks that this sequence folds to a similar structure uh then it has a higher likelihood of being correct than like you know just whatever the prior likelihood would be. Uh so you can take your sequence now and you can measure like how consistent is this structure prediction method with the structure that you actually predicted for that sequence. You can now compare your design to an independent model structure prediction.

31:05 · Uh and that became like a really good way of gaining conviction that your your design model was correct. Uh, and people kind of like game these benchmarks for a while and kept pushing pushing pushing. Uh, it turns out like it's easy to get self-consistency, consistent design of structures if all of your proteins look identical. There there are a lot of problems that this this creates, but then people started adding more and more on top of this.

31:27 · Yeah, that is a that's an interesting point that I I think some people have acknowledged in the community. So, how did you solve that? Yeah, you can see that if you sort of use your oracle and also your sampler at the same time, you eventually will converge. Um, what do you do to stop that or to convince yourselves that you're doing something valuable?

31:45 · One of the nice things about structure prediction methods is that usually you have some calibration and how kind of how confident the model is in its prediction. Uh, it turns out these models, they can give you a pretty well-c calibrated uh, confidence prediction. So rather than just say this

32:01 · is what I think the structure looks like it'll say this is what I think the structure looks like and kind of like here are the parts that I'm not really certain about uh and you can kind of aggregate this down to like a single scaler uh and typically what people do is they'll look at like okay like not only how self-consistent am I how much does this independent fully model even like the structure that it output uh so

32:20 · that was one way of early on I I'd say to like just gain confidence and then like another thing that people often do is they'll look at like the diversity of their generations because again you could have a model that's perfectly consistent, gives you great confidence predictions back, might be the same structure every time, like same sequence every time. So you also want to see like, okay, how diverse are the solutions? How many of these new problems can I solve in a sense? If I had a lot of whole lot of money to validate how would you do that?

32:45 · Can I go and you know do crym or something like that and try to figure out the structure? You know, sort of get some ground truth on that.

32:56 · It's more that the feedback loop is really slow. So you can validate a few structures like this but uh it might take months um and it's it's just not like a very scalable direction. So I think that's like a problem for the field as a whole. And I think people are spending a lot of time even like especially at Chai I think thinking about how do we validate these these problems at like bigger scale. How do we you know basically increase the throughput of our validation or increase the cycle time because if you're waiting months to figure out hey was my model correct? Like it's just it's hard to iterate in a research environment that way. The good news is that this is getting a lot better, right?

33:26 · Like there's a whole network now of wet labs that you can work with that will run, you know, these assays, these experiments and tell you things about say, you know, does your protein that you came up with bind to its target?

33:37 · Well, and so, you know, thankfully we're not at years, right? We're down to like weeks, which, you know, not as fast as like LLM land where you can just, you know, scale up an eval with and throw more compute and get results back in hours, but, you know, fast enough to where you can start to recursively self-improve. And you know I think we we also spend a lot of time like you know figuring out what are the metrics that we can compute you know in silicon like on the computer that are predictive perhaps of lab success but you know you see your question about cryomm.

34:02 · Yeah I mean also you kind of have to measure the the structure and as you know that's like so expensive because you have to kind of freeze the protein and shoot these electron beams at it and see how they bounce off. I remember there's like this really funny anecdote. We'll see if I can share it. But like the um you know the paper uh in Chiu we actually you

34:20 · know did that we took some of the you know the the proteins that the the model predicted and and ran Cryom and we got the results back and we're like wait the the results look wrong because we we had overlaid the kind of prediction over the point the the electron cloud the point cloud we didn't see any difference. And point being like we're we're getting to the point now where these structure prediction models are within you know a few angstroms or less of the actual atomic positions that you validate.

34:45 · And in this case it was a 0.33 angstrom error which is 1/3 the width of an atom and we were like this this like can't even be right. Like clearly they just sent us back the wrong design.

34:55 · They just sent us back our design.

34:57 · Yeah. Exactly. Yeah.

34:58 · Did you check for data leakage? Uh yeah, in this case like there there were no so like we actually chose these targets uh specifically like to have no known antibbody binder. Uh so like if we did get hit like it was definitely the first antibbody hit to this target. Yeah.

35:12 · I think that's one of the things I didn't realize about biology was like just how much of it is literally feeling around in the dark and that's not even a metaphor. You literally can't see like how these things look, right? So structure models are so so huge because now you can okay you can actually predict within an atom you know how these things look and that enables you to then do things like chai 2 with the design models.

35:33 · This to me is AI for science is [clears throat] one of the cornerstone problems right is that you don't know you fundamentally don't even know how to measure your problem in a lot of cases. So it's very difficult to validate.

35:46 · Yeah. So you you you're getting these sub angstrom predictions with chai 2.

35:51 · Chai 3, what why chai chai 3? What's better or what?

35:55 · Yeah, I think like with with chai 3, so like honestly like there was a chai 2, there's a chai 2 and a half, there was a chai 2.7, there was a eventually a try 3 and like each time we saw better and better performance. Uh and I think like the the main thing with try 3 is like we look at chai 2 and like we look at the targets it could solve. There was like a lot of internal discussion uh after try 2 like hey we made like successful molecules binders to half of these 50 targets.

36:20 · What about the other 25 you know what can we do to make those better and then like you know we were split we're like all right should we like study these targets that we missed and like figure out exactly like are there properties of these that we can look at uh or should we just bet on the models uh like will the models just get there if we put more time into like you know just be bitter less impilled in that sense and just really bet on the models getting better. Um, and we we definitely took the the latter approach like we bet on the models getting better and we just pushed as hard as we could on that front.

36:48 · Scaling up the model, the data, whatever to to just build more accurate models.

36:56 · Yeah.

36:56 · Is it accuracy? Is that the main thing?

36:58 · Is it binding affinity? What do we So, I think binding affinity is a big one. Like you you you you can't just bind weekly. In order for this to be like a useful tool, especially for our partners, we need to start producing molecules that are like at or very close to therapeutic grade, which means like they have to bind really tight. They also have to be developable. They have to have like all these nice therapeutic properties. And developability, I think the we talked about he he mentioned Chai 2.5, right? Which we released like a few months after CHI 2.

37:24 · There was a study we did on the developability of the molecule which um you know for the audience like obviously the molecule has to stick good and stick tightly but you know there are these other properties you care about and to use the non-biological terms right is it is it safe is it stable is it easy to manufacture does it you know self- aggregate um and uh we've we've been pleasantly surprised at you know how how much we've been able to climb and push the performance um in those areas it seems like one of the reasons that you want to do antibodies because the developer ility.

37:56 · Yeah, you get a lot for free there, right, with that antibody framework.

37:59 · Yeah, it's interesting. I mean, to me, uh there are many structure prediction molecules out there, I mean, uh models out there. I feel like the it's these other ancillary factors actually that are going to probably be the most impactful in the usefulness of a of a product, let's say.

38:18 · Yeah. Right.

38:18 · Yeah.

38:18 · Absolutely. The nice thing about structure prediction is uh there is a ground truth that you can compare against. for design, you don't really have that. You're like, "Here's some new like disease molecule. Give me a binder for that." And like if you want to know if this thing really binds, you you have to send it off to the lab and wait a while. Uh for structure prediction, you can be like, "All right, the model hasn't seen this sequence before. It's never seen anything close. Uh does it actually like fold up into the correct shape and we can just kind of hold that out of the data set and check?"

38:43 · Uh so I think I've always thought of structure prediction as this really nice speedrun kind of benchmark to like validate ideas on.

38:51 · Right.

38:51 · Sorry, I I didn't mean to say I meant uh you know sort of structural models in general. Yeah. Um but yes, exactly. So maybe uh we can um talk [clears throat] a little bit more about getting start getting into the the product side of things. Thank you for coming. [laughter] I actually I mean like like I said, I really think this goes throughout not only for um you know sort of structural models like this but also virtual cell and whatever. it's really um the all the other stuff around the the drug development process that is going to have the biggest impact.

39:22 · So can you talk a little bit about that?

39:25 · Yeah, I think that's actually a good thing to talk about after CHI 2 because I think CHI 2 is where it started to get really fun from a product perspective, right? Um I think with Chi 2 we we crossed the threshold of usefulness where after we you know released that paper we had a lot of you know you know pharmas and biotechs approach us and say hey this model might be able to do some stuff for us like can we use it and then we're like oh man like we we should build a product right we should build [laughter] something to let you use that model um and and that's right around when I joined and there was sort of this you know mad mad buildout to both you

39:56 · know build the product which we can talk about the shape of and also go and secure the compute actually so we can go and serve those models to our partners.

40:03 · Um, and you know, I think another third piece there that was really interesting is, you know, around security and IP, right? I think we want to be a very neutral platform that anyone can design medicines on, but as you guys know, like pharma is this notoriously IP sensitive industry, right? And I think when I joined a lot of people told me this can't be done like they're not going to put their data in a platform and like have all their new medicines be generating out of it. and having a bit of a background in security helped a bit.

40:29 · Whereas like no actually if you like just are really aggressive about how you like segment data and set up like single tenency where you're like almost deploying a separate version or separate account in the product per customer. Um you can actually like build a platform and then go and ship it to them. And so um you know through the

40:47 · summer of last year we started doing that right and you know we'd been working with uh you know or talking to um to Eli Lily and you know they were you know um one of the first uh partners to really uh work with us closely on that kind of uh you know made that V1 of that uh that design suite right that you can use to to engineer some of those molecules on and you know maybe it's worth talking a bit about that design suite right I think um you know I think I we have these really really powerful models now, right? That can do like all of these crazy things if you condition them in the right way.

41:18 · If you kind of give them the right context about, you know, the structure that you're going after or maybe the constraints around the model, right? Like, hey, I want to design an antibbody that hits this GPCR protein, but, you know, doesn't collide with the cell membrane and also targets the specific epitope on that as well.

41:35 · And, you know, we looked at and we're like, I guess we could put a chatbot around it. that'd be like really easy to talk to, but like really like you're trying to build something almost very visual, right? And you can finally build something really visual with some of these structure prediction models. And so if you kind of look at the Chai product, it it looks a lot less like a, you know, a Chad GBT and a lot more like uh Autodesk or or Solid Works or or Figma, you know, if you've used those things where you can kind of load up your molecule. There's this almost like Photoshopesque like design suite.

42:03 · You have this equivalent of a paint tool to kind of paint your epitope. if there's a equivalent of a contentaware fill tool to kind of get your uh your binders generated from chai. You of course have a lot of the scientific analysis and plotting and whatever to understand uh the results of the of the models. But um we've just been surprised at like how much complexity is actually just in like doing that right um so that you kind of don't shoot yourself in the foot when then you're then prompting these uh these models to to give you advice.

42:28 · So, are you sitting with people who are designing these antibodies, you know, and in like and then they're complaining to you or whatever?

42:39 · Yeah.

42:39 · How does that how do you convince med chemists to to use your tools because med chemists hate AI tools like notorious like I don't want to touch this thing or like I don't understand it and they will not touch things which they do not understand.

42:53 · Well, it helps a lot to have the models working really well, right? So when we uh when you know when we had the results of chi 2 and chip 2.5 I think you know that's enough of an activation energy where you know uh pharma companies and the scientists within these companies are like oh let's try it actually can chai can you guys just try running the model against a few of these targets and let's look at the results and then we do that and the results are good and they're like okay let me let me try to get on that product and let me try to use it. No, I I think Pharma is like incredibly pragmatic actually. Like I I've been very impressed with everyone that we've we've worked with so far.

43:23 · Uh they're they're very like I was saying pragmatic about this and they're like they're they're willing to be proven wrong and like I actually don't blame them for not trusting the models. Like I have used these models and like they like rightly so. Like I I I am pretty skeptical when I like see a new release.

43:40 · I I always have been. Uh, so like you really just like need to show them the proof and like they can give you this target that they are interested in or maybe it's more of something they've worked on in the past. They probably don't want to like share IP right out of the gate, but they can be like, "Hey, you know, I've had trouble with this particular target in the past. Let's see how how you guys can do on this." And then once you show them the proof, they like almost overwhelmingly are willing to accept that.

44:02 · I come from a cyber security background or you know have worked on security products before and those were dark dark years because you spend a lot of your time actually selling to people who are surprisingly not that technical. You think cyber security people are very technical in many cases they're not and it is this kind of like uphill enterprise slog to this very unsophisticated customer. I think we've been just pleasantly surprised I have by just how much I enjoy working with our partners and our customers. You know, these are scientists who have been spending, you know, 5, 10, 20 years of their life working on one target, right?

44:34 · Often in some cases, and they've studied everything about it. You know, they're they're very sophisticated. They're very smart, right? Um, you know, getting to collaborate with them is is is just a gold mine. And we learn a lot about how to make the product better. You know, there's this anecdote. We um you know a few months ago we were actually showing some of the the the results that we um from a target that with a a pharma partnership and um uh one of the scientists in the room like started tearing up and crying.

45:02 · Oh wow. [laughter] You really hit the head with that one and she was like we were like what's wrong? She's like no I've just been I've literally spent 10 years trying to get an initial binder to this thing and you guys were able to help me do it.

45:15 · Oh that's um and you know that that feels really special to answer your question. you know we you know there's of course the teams of scientists and computational biologists that we're working with within you know each of our partnerships there's also the people we have within the building right so we um I think one of the things that I really appreciate about chai is how cross-disciplinary it is like you know we have people who are maybe engineering experts and less bioexperts like myself we have great you know AI scientists but um or um ML

45:42 · scientists but we also have um a bunch of scientists that we we work with and and have have joined try to sort of help us both you know test the limits of the models right see what is chi 2 actually capable of what targets can it do what can't it inform some of the research direction there I want to add to that like um in in like the chi 2 days like we we kind of started with like a bunch of engineers and people had like AI bio experience we didn't have a hardcore lab scientist and like one of our first hires on that realm was uh Nathan Rollins who uh I

46:12 · think he he started working in the Baker lab at 14 graduated from Harvard at like 18 and got his PhD by like 21 or something like this uh in the Mark's lab and he was like super skeptical about Chai at first uh and then you know the results start to come in he's like okay this is this is kind of interesting like this could work and then like once the chai 2 results came back he was like I need to bulletproof this like nobody celebrate yet like all this uh [laughter] so I think like it's it's been really nice to have that level of rigor to just have people who have

46:41 · really like they've spent the time in the lab they've designed proteins themselves they've literally in in the case of like Andy led several therapeutic programs, brought drugs to the clinic themselves, uh, and like we have all these people internally at CHI just like using the product and like really battle testing that. So if you don't have your own platforms, right? I mean, so you don't have your own programs, right? You're pure platform, your or partnership model, right?

47:04 · Yeah.

47:05 · How do you battle test something if you basically aren't you don't have a use case where you have to continuously push it forward or if you are just pushing things forward? when you just end up with your own candidates if you're successful and then what do you do about that? I mean we have benchmarks of our own internal cases right you know there's a set of targets that you know have are known therapeutics right that have known therapeutics against them there's a set of targets that we pick to sort of push ourselves right and so we're constantly refining that set and

47:32 · adding to it and that's what that internal science team that we have helps with right is expanding that and almost running the experiments to try to get initial binders there we don't care about going and developing those drugs like we just do that in service of validating and making our models better and then of course there's a loop with with our partners too.

47:48 · Would you consider yourself hit discovery or are you do you I guess using some jargon hit to lead lead optimization like where do you live in this and you know hit discovery might be like one part of it which you can do hit discovery but the the later uh the other parts of this are I think often times much more bespoke and kind of special. I mean, how do you balance that? And it's it seems much more much more difficult to me to be general than it does to to solve general lead optimization than it does to solve like a discovery.

48:19 · I think ideally like we we really want to be able to rather than think of this as a bunch of stages. I think part of the reason why we think of it that way is because the initial molecules are usually like not good enough to be drugs. Um, and like really like we're kind of at the inflection point now. We're really seeing this internally at CHIA where the models are getting pretty close to like producing molecules that could eventually or like are very close to drugs.

48:43 · Um so we we try not to make too much of a distinction between okay hit discovery, lead optimization, all of the different parts of this kind of pre-clinical pipeline. Our our like you know the the light the north star is to just really produce drug-like molecules

48:59 · straight out of the models. of course this is going to be hard and like there are going to be like tons of roadblocks and like you need to be able to like actually prompt the model to do this you need the whole RL stack to like learn different properties things along those lines but I think it's very achievable yeah and I think to add to that right yeah this notion of target discovery and hit discovery and optimization where each of these has a gate and takes a few months to a few years is this very like waterfall model right where the cost of

49:24 · trying things and getting things early is very expensive but I think to what Matt's saying right if you start to get in a regime where you can have models give you really promising candidates, you can start to make that look a lot more like a loop, right? It's it's akin to like becoming more agile in software development. Internally, we kind of have two, you know, north stars, right?

49:41 · Um and that's at first pass, they almost sound like contradictory, but you know, the um you know, the north star in research is to start to denovo one shot, you know, better and better and better medicinal candidates that are as close to being ready for, you know, the next phase as possible. Um but you know also within product we we do want to sort of expand into whatever these iterative workflows look like right where maybe I get a binder I get some results from the lab I'm using that to condition my next run of the model.

50:09 · Um, and I think, you know, they sound contradictory, but I think they're actually not because I think what's going to happen, you know, the research is going to get better at identifying a denovo candidate for like a specific class of drugs, right? Say like antagonists, right? Like blocking things, right? Little bit easier maybe.

50:25 · Okay, we can get to a state where we can one-shot pretty good drugs there. But now the next problem is like agonists, right? Like how do you reliably oneshot hitting a switch like on a cell, right?

50:35 · Or by specifics or ADCs, right? And I think, you know, there's kind of this uh levels of abstraction that we're going to have to climb with the product as like the models get better. One of the things uh I was I got I got very existential like a few months ago cuz I was like, man, all this stuff we're building in the product to like visualize molecules and do this like maybe I'm just going to have to throw it all away when like Matt ships like Chi 4, right? Um but you know, I think that's that's kind of the reality of like building products now, right?

51:03 · You're actually using them less as an end in and of itself. Like maybe you'd have built software that was supposed to last like 20 years. Now it's supposed to last maybe one year, but it is the bridge to deliver value and kind of enable the research that then gets you to the next thing. And so I'd imagine we're probably going to rewrite our product at higher and higher levels of abstraction, right? Like maybe like right now we have something a little bit more akin to cursor where you're, you know, inspecting the molecule in the same way you're inspecting the code because you really need to verify like the bonds that are forming and the the the properties of the things that you're getting.

51:33 · But then, you know, you get to a point where that stuff is solved enough where now the product is actually just helping you orchestrate these like campaigns of hypotheses, right? Or maybe you have like one target and you're like orchestrating a bunch of different epitope choices or whatever against that. And then maybe you're going up one level of abstraction where you're now doing a whole campaign against all of the uh targets within a pathway, right?

51:55 · Um, and uh, I think what's really exciting about that is if you if you have like these really good primitives for structure prediction and binding and design and you can kind of compose them, then you can start to just like grow into like the outer loop of science, right? And then, you know, maybe the thing runs itself and uh, you start to really get to some really, really, really cool drugs at the end of it. I actually want to push on what you just said about epitope prediction because I think a lot of people in the field would argue this might be the much harder problem than finding antibodies and binders.

52:23 · Where do you think that the state-of-the-art is in general and also with regards to chai in terms of epidote prediction and like is this a problem which has a reasonable solvable time horizon? Oh and also maybe can you define epitope prediction?

52:37 · I'll think of this at like some different levels. So the most basic level is okay, I have some disease that I want to target and what proteins are actually responsible there. Like actually figuring out biologically what's going on, like what should I be targeting in the first place with the drug? I guess once you figure that out, um it's kind of like a structural biology problem at that point. You're like, "All right, this like set of proteins is responsible and like what's going on there?" Well, this is interacting with some other protein that it shouldn't be interacting with. And conventionally, you'd just like want to block that interaction or something within anybody.

53:04 · Uh, but that's kind of where these proteins interact and like the type of interactions that you want to disrupt. That's typically like the epitope. It's like the actual site on the protein that you want to block. This is a ridiculously hard problem. Uh, I'm with you on this. This is like the harder problem. Uh, like just the amount of context that you need and like the global understanding that you need you need to get in order to like actually figure out what's interacting and how.

53:28 · But maybe let's take a few specific cases. Let's think about what about um SARS KV3 comes around or the new flu or whatever. What would you do there? I mean is that something that you think you could actually reasonably tackle in that case? Like yeah, you could just run a structured prediction model maybe and like see where the model thinks this thing will bind. Uh if it's highly confident in that, you might say okay here is like the site that we want to block. I think in general still very hard and even like structure prediction it's getting really good and like a lot of people think Alful 2 like solves structure prediction. Not really.

53:58 · Like alpha 2 got like I think 11% the multimemer version of this got like 11% of antibbody antigen prediction cases correct. That means 90% of the time it's wrong.

54:10 · Yeah.

54:10 · I mean but but uh alphaful 2 solved a certain class of monomeic proteins with MSA. Yeah. Yeah. So I mean the and that's the MSA I think might be the the key point here because MSAs are sort of the the the magic which makes it all work. It's like a it's a template in some sense about like what the structure should be and antibodies almost evolutionarily can't have a template, right? Everyone has to have unique antibodies accustomed to the things that they've experienced over the course of their life.

54:40 · Yeah.

54:40 · So, right. And ju just to clarify, I I had to understand this myself so maybe I can help the listeners who aren't familiar.

54:46 · An antibbody the whole point of an antibbody is it can identify new things that it hasn't the body hasn't encountered before. So the design of antibodies as opposed to other types of proteins is to the system is designed so that you can quickly recombine different components of it in order to um match uh proteins that are from unknown pathogens more or less. And so this is why you it's not conserved in evolution the way that other part other proteins are.

55:18 · Yeah.

55:18 · So so like back to the the epitope prediction problem. Um I I think it's still hard. I think like there there are a lot of cases that that are maybe tractable, but I think in general like if you want to discover this for a new target, uh still still a really difficult problem. Maybe virtual cell would be like the closest thing to state-of-the-art there, but that's still still a ways out. I wanted to dig in a little bit on the product cuz I there's something I don't understand about the economics of basically all all the structural stuff that's happening right now. And obviously a lot of people think it's very very valuable.

55:49 · So there's, you know, I'm not grocking something, but when you look at the cost of developing an antibbody, you know, it maybe is a couple million dollars, right? When you go from you you you've identified a target somehow and then you say, okay, I need an antibbody to match this and then I have to sort of optimize it in various ways and then maybe I try it in I mean with antibodies, you go to animal typically faster.

56:16 · If you look at how much does it cost to drink bring if you like are precient and pick the right target and the right technology to get all the way to drug it might be half a billion typically that $2.6 billion number is advertised over all the failures as well. So if you look at just the cost of that one success depending on the the disease maybe less but you know half a billion might be a good median number or something. So you're you're saving like a couple million dollars in a half billion dollar campaign. So why is this so attractive?

56:50 · I I would maybe challenge the premise a bit like in a few ways, right? like okay sure if you're trying to get an antibbody for like a very simple kind of target like maybe right but I think what we've been most excited by is our partners using antibodies in you know more sophisticated ways right like in for example in Chi 2 we showed like GPCR agonist activity right where you can really hit the switch on a you know on a

57:12 · cell um doorbell protein so to speak right in a very precise way very very very hard to do that with antibodies if you can't be that precise right so you're unlocking a new capability I would think about it as less like, oh, I'm taking the existing drugs that I can do and making them faster. I mean, there is some of that too, right? But it's like, no, there are just like, hey, how do you go after like better targets, right, that are, you know, maybe more precise, more effective, right?

57:37 · I think like also on top of that too is like there are drug modalities that you just can't discover with immunization.

57:43 · Like you're not going to design your like crazy multi-specific warheaded super intense formats. Um these are really things where you kind of have to design these from first principles. Uh even just uh with bio specifics in particular like both arms need to now bind different targets. Uh and you've kind of like have this multiplicative effect on your binding rate. So like uh if you have a one in a billion chance of finding a binder in arm one and a one a billion chance in arm two.

58:08 · Yeah.

58:09 · You're [laughter] not this just isn't going to work with the traditional approach.

58:12 · Exactly.

58:13 · I think the other thing I'd think about is right, you're not just helping your partner with maybe one drug, right?

58:18 · There might be a portfolio of of targets that or they're going after a portfolio of drugs that they're trying to make. And the nice thing about the platform approach rather than the we are developing individual drugs is we can sort of scale with them as they pursue more targets in addition to more ambitious targets.

58:33 · Right.

58:33 · So it lets you uh concentrate your your learning in a subdomain of that and so that you everybody benefits from that. Exactly. That's the but I Okay. So I didn't So what is what are some of these capabilities? You mentioned a few. Are there more that are really interesting that you guys are chasing?

58:50 · Yes.

58:50 · I mean we talked about like you know cross reactivity. We talked about selectivity. We talked about some of these like really interesting additional modalities with by specifics right? Um there's a set of things that you know our partners have been asking us for that we've been working on that I can't get too into because then that starts to reveal some of the the targets that they're going after. But uh I think that the point being you can just once you get precise like you can start to do some really really cool drugs.

59:16 · It's a new technology right? So like technology in in pharma means like how do you deliver your therapeutic and so this is maybe a kind of thinking about like carti is a technology right and and and so this is maybe a new technology in

59:32 · the sense that you can have these highly highly engineered right and that comes you know from the mission of the company is to really turn you know drug discovery from a scientific experiment to an engineering discipline right how do you sort of get to the precision engineering phase is for biology where you can start with you know almost declaratively define the thing you're trying to get and have the model fill in the gaps and get you that.

59:56 · So what is the biggest blocker from going from science to engineering?

1:00:01 · Oh man there's so many things like that's the thing about you knowity.

1:00:07 · Yeah. [laughter] Yeah.

1:00:11 · I don't even want to talk about this like the the amount of headaches like too late. You already know.

1:00:16 · Okay.

1:00:16 · Okay. So, like just just like when you're actually parsing like first of all, file formats for biologists. Like I just they just don't care. Uh there's like no standardized there there are standardized file formats. Are they the best? I I don't really know. But there's like also just like a lot of information that you want to pack in. I have this structure. Here are the people who solved it. This is the method I used to solve it. There's like a lot of stuff going on. And then depending on the method that you use to actually figure out what this 3D structure is, you might have like multiple copies of that structure. Part of it might not have really been resolved or you're like it could be here, it could be there. I'm just going to give you like both options.

1:00:47 · Uh so like the actual just parsing problem on the engineering side of like working with this type of data is like really difficult.

1:00:53 · This seems like something that LLMs can excel at though.

1:00:56 · They don't know all the edge cases often, right?

1:00:59 · This is more back to just like a simplicity approach. Like LM are very good. I will absolutely give you that.

1:01:04 · Then you're thinking about like do I really want to like should this function have 20 special cases or should we be like really uh principled in how we approach this and should we be I guess more of a opinionated opinionated yes like how opinionated should we be in how we do this? We want a strategy that's easy enough for humans to understand and like when we're reading through the codebase we really need to know what's going on here what are the potential problems and like sometimes that just comes down to looking at examples.

1:01:29 · Um but then I think okay once you've kind of figured out all the infra work and how you get data into the models there's then like scaling the model there's then scaling the infrastructure around the model to train bigger and bigger versions of this uh and that's like a lot of work that Neil and the product team actually yeah I mean that would have been my answer is the infrastructure part I mean um you know not to beat a dead horse but compute right getting the compute and using it in the right way is such a challenge you know it's especially for startups and this has been such a Yeah. Anthropic is single holding back science.

1:02:01 · Anthropic opening.

1:02:04 · No, I mean and and to that point like we um I mean they're also accelerating science, but it's like this weird totally like one of the um one of the things that I uh help a lot with at Chai is buying compute for the company.

1:02:16 · Worst job, man. I would not recommend it. It is very stressful. Um but you know, even September of last year, right?

1:02:23 · Back to you the hardware job.

1:02:24 · Yeah.

1:02:24 · Yeah. I know. Exactly. In the wrong wrong way. But, you know, September of last year, we started to really notice like things were getting getting tight, right? We were doing a lot of our inference on, you know, spot and on demand markets, and we'd have these days where you just like get these capacity crunches and we're like, okay, we should probably start to get ahead of buying some compute for oursel. And, um, I mean, I think everyone probably says this, but man, it was it was hard.

1:02:48 · Like I think I didn't I didn't realize how much of a power law you know this is right where you know there's there's say 10,000 you know B300 units that are shipping everywhere right the um the hyperscalers and the you know the the biggest uh the biggest um AI labs are buying 95 plus% of it right and then you

1:03:08 · kind of have the startups like fighting over the scraps um and I think the other thing that's really interesting especially if you look at these later compute versions right the um the the the Vera Rubins or you know the B300s like a A lot of this stuff has been built very like LLM for it, right? Like you have these, you know, systems with like huge KV caches where you have like 72 GPUs that are all acquired to talk to each other, right? And you know, obviously some performance gains there like help us, right? But like it's it's kind of interesting just how much the compute market has kind of gotten LLM pill.

1:03:38 · Um I think there's like a whole probably set of, you know, compute stack and inference optimizations and things that need to be made for this class of models. And you know, I think this class of models is going to be like just as big, just as impactful as LLMs, but it's almost like the comput like kind of doesn't realize that yet. Both in the capacity sense, but also in like the software stack sense. So, we actually spend a lot of our time, you know, even just like doing basic optimizations of compute to like get them to work better for the types of models that we have.

1:04:07 · Yeah, I know that that some structured models are you more recursive than than LLMs for example and so that um that which which changes sort of like the maybe the compute to memory ratio that you need and things like that. What are some of the like sort of cool or interesting optimizations that you've done there depending on the type of model? So like we can go back to like a try one type model.

1:04:29 · In that case we're following the fold two three architecture and there you're like rather than doing attention over like this like normal sequence representation you're in a sense loosely doing attention over this pair representation.

1:04:44 · So you can think of this as like a sequence of length L squ uh rather than like typically length L. If you're doing attention over that the way that you actually batch this up it ends up being L cubed. Now you're you're in like a pretty pretty heavy compute regime. Uh so the amount of flops that you're putting into every token stays it's pretty high. The amount of memory that like the memory bandwidth uh overhead of just transferring that uh from like SRAMM to whatever that's a real bottleneck in these architectures.

1:05:09 · So like even something as simple as like a layer norm uh can can take a long time actually like that can be a significant amount of the compute that you're using. Uh so I think like on our side we've spent a lot of time just like optimizing it and engineering taking engineering very seriously so that like these operations are you know at least better.

1:05:28 · We're always looking at like how to new chips perform compared to the older versions. Sometimes that's even different for training versus inference and like of course Neil knows this really well.

1:05:37 · Well so so there's you know what you're doing on the individual GPU and then there's like how do you like orchestrate fleets of GPUs right? and you know uh you basically shard your computation right and so you know when you're designing a molecule on chai it's not necessarily like one call right it's a lot of a lot of GPUs being thrown at the problem right across um across a lot of compute and um actually I I would say

1:05:59 · that one of the hardest things to get right in in software engineering is durable execution are are you all familiar with that term I can I go on a little ultimately like if you're like computing a lot of data you know model calls across like a very wide set of infrastructure, you always run into these problems where like some part of the infrastructure is flaky, right? Like maybe the bucket you're grabbing your data from like goes down or like your database has a blip because there are like too many transactions against it or your like GPU errors out, right?

1:06:27 · I've been at companies before where you like spend so much of your time just dealing with this right? like you you're basically putting like all of these cues and like all of these retries and you're like duct taping things together and you have a and it becomes this mess where now what used to be like a ideally a pretty simple like computation that's just distributed. You're ending up spending like 95 plus% of your time on all of this queuing and retry stuff, right? Um we're huge fans of this company called Temporal.

1:06:55 · Basically, you know, there's this idea like look, if you're just trying to get something a really long running job to run at the end of the day, what do you need? You need a queue. You know, you need your flaky thing like pulling off of the queue. You need some retry logic to put things back on the queue if they fail, right? And then you need some whole like orchestration system to just like tie all the cues together and monitor them.

1:07:18 · Uh what's really cool about Temporal is like this is a a tech a company that's kind of invented a framework for doing this. And um one of the I think one of the technical decisions we made early on that was very helpful was to run as much stuff as we can on temporal right so whether those are um you know calls out to the database from the app right to make sure the database transaction goes through without failing okay let's have side effects like sit on temporal so that they get retrieded smartly without us having to like write our own Q logic right or things related to model calls or things related to orchestrating really long data pipelines.

1:07:49 · Point being like, you know, one of those primitives like just like, hey, you need to get durable execution right so that you're not stuck in like retry hell. Uh a really deep like engineering thing that like you wouldn't realize if unless you for like me and Jack, you've been like burned by this like many many times before. Um and I think like we're at this state now, right, where we've you know, we've raised another $400 million.

1:08:13 · Uh I have to go buy another compute cluster. like you know like we're going to have like really really really large runs and and inference and and and training sets. And so um getting those foundations right is what's actually going to let us do more ambitious things. And to kind of answer your question, I actually think that's a lot of the bi the the the bottleneck to making uh making biology more like engineering is just like having the right engineering primitives.

1:08:36 · I have an analogous tangent on the model side. Actually, one of the things that's uh kind of nice about those problems is they're like super visible. Uh so like at least you know like hey this this crashed this failed for us we just see like loss curve didn't go down or like we see weird gradient behavior or whatever. I think a lot of these same principles like you know engineering first that also applies on the research team. One thing that I like to say is kind of like complexity and being bitter lesson pill they're like fundamentally at odds.

1:09:02 · For example I think like alfold 3 I might get this number wrong but I think it was like 23 subm modules and at that point that's a really difficult system to optimize and study. you're like, "All right, what happens if I change like if I tweak this thing in subm module 30 or like 21? What what happens to the whole system?" And you can always think, "Hey, we can make this better by like adding module 24, but like should you or should you think about just like removing things and lowering that complexity down?" But I think that's like a pretty fundamental thing at Chai is just like the engineering culture and just being like very simplicity biased.

1:09:32 · Have you all seen the picture of like the SpaceX engines? It's like Raptor 1, has a [clears throat] bunch of pipes and like Raptor 2. We have a picture of that like on our office [laughter] wall cuz I mean it's just true, right? Like how do you delete delete delete more things?

1:09:46 · Yeah.

1:09:47 · But the only way you can accomplish that is I mean the reason alpha 2 and alpha fold 3 worked they were small models relatively speaking. They were very comput intensive but they were very data efficient.

1:09:58 · Yes.

1:09:59 · And like the there was inductive bias after inductive bias brought in by human intuition and probably like hard hard fought experience. Mhm.

1:10:09 · Um it was they're incredibly efficient.

1:10:11 · If you try to knock down those things, you know, they're not like a house of cards. Like everything is a incremental improvement on top of it. In order to get beyond that, it seems to me like you really just need new sources of data. Uh you need to at least treat data fundamentally different in a way that is much more efficient. Um I I mean I mean I'm actually kind of surprised to hear that you have scale to that degree because I suggest that you're doing something very different from what the community is think the way the community is thinking about it.

1:10:40 · I don't know if you can comment about that but we're pretty first principal people like the whole research team at Chai. Uh except for me and Kevin really um like we're the only people with quote bio background even still like we're we're pretty far removed. So I think like we we try to like look at every problem as a core ML problem. We try to think of like what's the analog in other spaces.

1:11:02 · So like um even for image models like CNN's were built to process images. So like images should be looked at in patches like that was the nice inductive bias there. Then people are like well you can just kind of tokenize this thing throw it into transform and it's going to work and like it did end up working.

1:11:16 · Uh even like on a relatively small data set but I think um for proteins in particular it is really hard. There's not as much structural data. there's a ton of sequence data and like that's one of the unlocks for like ESM working. Uh you can get that to just run on a transformer. If you try to do the same thing with like experimental structure data, good luck.

1:11:33 · You need I mean there was the there was that Apple paper where they distilled on the dis fold which it was actually really cool that you could distill on a very large data set and you could get you know good signal but you know it didn't generalize at all because it wasn't reasoning. It was really pattern matching. Like one of the things these like triangle layers you were talking about for example uh they do have a very nice inductive bias.

1:11:55 · Maybe it's not the triangle inequality like the paper origin really proposed but it's a clean inductive bias and it unambiguously is like one of the the things which made it work and it just comes at a huge cost.

1:12:08 · Yeah.

1:12:08 · Yeah.

1:12:08 · No, I think that's that's definitely true. These layers are pretty costly. Um and like that kind of limits what you can do with the architectures.

1:12:15 · They're not like not only are they like costly in terms of compute, they're just like not efficient on modern GPUs either. You have small hidden dimensions, large sequence dimensions, like it's like exactly the opposite of what GPUs are designed to process. One take away from like triangle layers is you're kind of just trading off parameters for compute in that sense.

1:12:33 · Like that's like one mental model for thinking about this. uh I might want to like throw more compute at the problem and just trade that off for parameters cuz like I won't be able to hold as many like I can't literally store these you know large pair representations and still do normal attention uh so I think

1:12:49 · there are fundamental things you can abstract from the ideas like alphafold um but you can kind of just like tweak these and start building off of them in your own way it sounds like you have quite a bit of research um like fundamental research going into this direction for I guess audience looking for a nerd snipe and uh ML engineering for new problems probably something very uh it's a very uh different research direction than a lot of the communities going in.

1:13:14 · Yeah.

1:13:14 · Yeah. I think what we built at Chai is like it's it's very unique in a lot of ways uh but also very tied to like what CoreML is is good at kind of what I was saying before like we try to map every problem into like a core ML problem. We think you know how would you approach this if if it were an LLM or something like that. Um, but yeah, like at the end of the day, we really really value simplicity. Uh, and we we really encourage people who don't have a bio background to like not be scared of this stuff.

1:13:41 · And I think that extends into the product too where you know there's a balance to be had here, right, between like how general do you make the product? Like do you build a cross reactivity workflow and a selectivity workflow and a by specifics workflow or do you all say no like let's make the

1:13:57 · model general enough to say I'm going to like condition on arbitrarily binding or avoiding something and then you just have a very general like screen in your CAD suite where you can say hey I just want to avoid or bind to these parts of these different structures right and I think um you know kind of like the the ML team like I don't I don't have you know a formal bio background most of the the product and platform team doesn't have a formal background either Now, there's some amount of like maybe regretting my words that I'm gonna [laughter] have, right? Because I'm sure there are, you know, a million nuances and, you know, I don't want to come off as, you know, too too brash or naive there.

1:14:27 · Um, but, you know, I think I think it's sometimes it's helpful to not be burdened by like all of the, oh, these this nuance and this nons and this and you can you get to kind of bet and be maximally uh general because, uh, you know, that's kind of what we're seeing in the research. You can the models are very general that lets the product be very general.

1:14:42 · I'm thinking back to like in my in my CS theory days my first adviser was like uh we're working on some problem and we we needed like a polinomial time algorithm for something uh and he he would always tell me like never underestimate the power of polomial time like this is basically

1:14:58 · like you're allowed to choose like whatever exponent you want and my first paper was an n to the 20th time algorithm [laughter] for this problem and I was like Andy I did exactly what you said he's like wait a minute I didn't mean it like that yeah but I think like [laughter] it kind of like you can really help yourself like you can free yourself a lot when you're like, "All right, I can kind of do whatever I want and then kind of simplify it later." Uh, and I think that's really like a pretty fundamental way of thinking about things that we we leverage a lot at Chai. The space of binders of protein design and binders in general is actually a fairly crowded space.

1:15:28 · I I'm curious about what your general outlook of the the field, the industry is. I mean, I can go back to like some anecdote. I was it maybe Nur's three four years ago, right? The one right after RF diffusion came out. I was talking to someone in the Baker lab and they're like, "Man, I just one-shotted." I don't think they even use one shot.

1:15:47 · One shot wasn't even a term back then, but they was like, "I just got picolar binders out of RF diffusion and just like threw in the cryo." Great. Right.

1:15:57 · It didn't seem like that just solved the problem. Like, it's not like, oh man, now every Yeah. But there are lots of people who I think have seen that you can actually do protein design at least in some categories quite well. I'd say like is it mini proteins or mini binders? Um ironically nanobinders are actually smaller than or larger than mini proteins or maybe like a little bit harder. Antibodies are typically considered even harder. But there's this like is this something which can and will be commoditized at least in some part.

1:16:29 · How do you compete? like where does this where do you where does the field go from here?

1:16:34 · I mean I think the answer is it's kind of all of the above. Like I think there probably will be some commodity layer for for certain types of modalities or drugs, right? I think at the same time we're going to be able to do even more and more and more ambitious drugs and you're going to it's just like what's happened in LLM land, right? Like you have your your open- source models that are maybe general and helpful for some things, but people are still buying Frontier models, right?

1:16:56 · And actually if you look at the amount of value captured it's actually the the closed source frontier models you know the whole pie is growing but it's growing so fast that even as the open uh source models like share expands the the uh the frontier models are still able to capture the majority of the value. brings you kind of if you're using an open source model on your day-to-day, right? And what are the reasons for that? Right?

1:17:19 · One, like if you have, you know, more intelligence, you're going to go after harder tasks, right? I think if we have more, you know, intelligent uh biomodels, we're going to go after more more crazy biotasks, right? Um but then also too, like I mean a lot of the reason I don't use the open source model is cuz like you know, I don't get like cloud code, right?

1:17:34 · I don't get like cloud, you know, I think there there's like a product layer to be built that is uh just as important as the model layer. um we learn a lot from our partners and you know the people in the building as well just like what are the really uh tough things that they get stuck on using the models right and some of them are like you know the dumbest things right like um you know I want to be able to better visualize this piece and like focus on that and some of them are actually like very sophisticated things that we then have to build some like pretty vertical product for and look maybe in the fullness of time like AGI like oneshots

1:18:06 · everything and doesn't matter but I think there's quite a bit of uh of time until we we get there right and I think um the the product uh makes a huge huge difference for that. That'd be my answer. I mean, you probably have a more model forward answer.

1:18:18 · No, like I I think like like biology is slow uh which is like one kind of nice thing and there's like not that much labelled data. Uh so like you could take all the publicly available sequence information out there that might give you a good base model, but you still need some measurements on that data that's still pretty timeconuming and then you need to like iterate on that.

1:18:35 · Um, so I think there are even just data blockers there uh into unlocking like if we really want to do this uh zero shot design candidate start generating molecules that are almost ready to go in into the clinic. Uh I think there's more to that than just like you know AGI might not solve that right away.

1:18:51 · I think there are definitely like some technical blockers there right but even in the space of you know specialist companies I mean I'm not going to like to start naming them but there there's I think I don't know probably 10 15 protein design startups I think that the two things which it sounds like Chai has gone on is like one all-in-one product and two you are not trying to do your own platform if you don't have your own data mode you know is that going to like help you win out

1:19:17 · in the end or is that going to be a you know a blocker I don't I'm just I'm just curious about Yeah, that's that's a great question. Yeah, so so try definitely no plans of like starting a pipeline like we take the partnership model pretty seriously. Uh and we I just like from a personal stance I love the incentive alignment between like you know we make the models better, the partners succeed more and just like you know that iterates on itself. Um so like I think that's like a pretty unique part of Chai is like one just being able to partner with a lot of people. Two getting like the feedback on the product.

1:19:48 · So like you know knowing that it's very real. this is in like like legit big pharma hands and like they're actually running campaigns on this stuff. Um so I think uh it's interesting we really have to be model forward model focused like we need to keep delivering value. Uh so that puts a lot of pressure like on the research team the product team first of all to like to serve these things the research teams always shoot for like better and better versions.

1:20:10 · The way I think about this is like if you're a bitter less impilled forward kind of like thinker or company uh then there kind of comes a certain point where there's a lot to do on like both the model and data side but I don't think either is exhausted. It would be stupid to say like we don't need any more data but also be stupid to say like the models are stuck. We only can like use data to solve these problems. So I think there's like tons of room to grow on both sides. We're taking like both very seriously. And I would also maybe push back on the no data mo premise, right?

1:20:40 · That'd be kind of like saying, hey, like all the enterprises that work with enthropic, like you're not letting like enthropic train on your data. So like you can't like build models that are good at enterprise workflows, right? I think you know, one we are investing in this, right? You know, there are ways to turn compute into data and get get more and we're we're doing those, right? But then also two, okay, what is the kind of data that you're trying to get, right?

1:21:00 · And I think what what is kind of cool about you know working so closely and supporting so many of these partners is we get to really learn about um you know what is like the stuff that that would be helpful in research right and so rather than doing research in a vacuum you know based on what would hypothetically be cool we're we're able to sort of kind of do informed research based on like you know what our what our partners have just been very organically asking us for help with.

1:21:23 · I see. Do you I assume that you aren't allowed to train general models based upon your partner's data. Do you train spec special specialized models for like does is there artist model and a fiser model?

1:21:34 · Yeah, I mean like a lot of these a lot of these deals um you know and and this is all public right we we are working with them to you know uh train or fine-tune a version of our model for them and I think there's probably like so much more we can do there over time.

1:21:47 · Um my brother started a company called Applied Compute. Great company. They're kind of doing this thing for, you know, design for LLMs, right? And helping enterprises really understand the the value of their language data and do that for specialized tasks. I think there's a whole world where we could potentially do that for biological data.

1:22:02 · What what is the value there? Like what what is the lift that you get from using their data? I mean, is it just that it's more data or is it more that there it's specialized to a problem? you know, they have they have a lot of like scientific, you know, data that they're getting from experiments that can maybe help uh our models do better in like particular classes of of candidates or targets that they care about.

1:22:23 · Yeah.

1:22:23 · I mean, even something as simple as like they might just have some preferred way of doing things uh that might not be like native to the chai model uh and they can like you know kind of like ask the product team and in a sense it just be like hey we like you know our our designs have property X can you make sure that they have those? Um so I think like even things as simple as that um they are actually have like a pretty big impact for them.

1:22:46 · Yeah.

1:22:46 · So I mean this this goes along with the a a pet hypothesis I have that all AI companies and especially bio and scientific ones are actually consulting companies. Pharma I think is particularly the case because you're developing a new drug right it's almost

1:23:05 · by definition new right so like the existing stuff has to be customized in many cases right unless you're doing something that's just reiteration of old stuff but a lot of the big farm are are pushing the boundaries of science yeah I mean certainly like we aim to make the models very general we aim to make the product very general we aim to make it powerful but yeah I mean there is integration work right with every with every customer to answer to your question, you do get some defensibility just by doing that, right?

1:23:30 · And um I think what is what is nice about building, you know, trusted relationships with these partners is hopefully, you know, if we execute really well over the next uh you know, the first year, then they'll continue working with Chai to to do more ambitious and and more more drugs past that.

1:23:46 · I mean, it's going to be hard to switch, right?

1:23:48 · I hope so. Yeah. [laughter] Just getting the security.

1:23:52 · Yeah.

1:23:52 · Yeah. Like maybe maybe one other interesting point is like if you think of this like on a per token basis. I don't know if there's another domain where like the downstream value of a token is like as valuable as it is for pharma like you know you're think about like the actual drugs that come out like these can be like multi-billion dollar assets. In the case of GLP1s I think the two GLP1 drugs combined are like maybe a trillion dollar asset like Yeah. I mean up until I think 3 months ago right GLP1's like total revenue was more than all of the AI labs put together. Yeah, I don't think people realize that. Like I didn't realize that it's crazy, right?

1:24:25 · But yet the market way lower. It's like crazy how relatively speaking the market is.

1:24:30 · And you know, I didn't realize how much of like a VC business, you know, uh, you know, pharma is in, right? They're in some sense like taking really ambitious bets. Uh, you know, I think one of the things that was really cool was, you know, is like if you study the history of Silicon Valley, right? Like obviously people think of Silicon Valley with software but you know in the ' 80s one of the one of the biggest uh venture outcomes one of the first ones was uh was Janentech right um and because it is

1:24:55 · such a VC model right you get the string of tokens that uh can then give you so much value downstream just just general shout out to outposting's uh blog blog series about like finance and uh funding and uh yeah really fantastic yeah before that I knew a lot of those points but I did not realize just how deep that rabbit hole went. Yeah, it's uh Yeah, I mean it's um maybe the big single biggest problem in bioarma is actually just the funding model.

1:25:23 · There's also have you have you heard of Aram's law?

1:25:25 · Yeah. Oh, yeah.

1:25:26 · Yeah. Yeah.

1:25:28 · More backwards.

1:25:28 · Yeah.

1:25:28 · More Moore's law backwards. So, it's like uh in like compute, you know, it's kind of scales uh so you have like this nice exponential scaling log line scaling of compute near the exact opposite in pharma. So, like the cost of actually making a drug in pharma is kind of like increasing exponentially. like the amount of money put in per drug is growing at kind of like an exponential rate which is it's pretty interesting to see this. Yeah.

1:25:50 · Which guarantees at some point the marginal return on a new drug development will be negative. Exactly.

1:25:55 · So unless someone I maybe chai figures out how to you know fix this.

1:26:01 · I think that we might be on the verge of sort of flipping some of these bending the scurve.

1:26:08 · Yeah. just to double maybe belabor the point but that pharma and VC fundamentally both are optimizing a portfolio. Yeah. Right. And I think that's the that's the connection there.

1:26:19 · Yeah.

1:26:19 · Thinking of pharma as like sophisticated capital allocators, right?

1:26:23 · Where they have these this portfolio of targets and they're allocating between them. I think uh that was a big reframe for me and I think I think we will just see more of that in the future, right?

1:26:31 · And hopefully they can take you know in the sense the VC taking riskier bets like hopefully pharma can take riskier bets and pursue really really cool drug targets in the future.

1:26:41 · That analogy is actually like uh one the the kind of like VC type investorish model. It's like actually how we think a lot about research at Chai as well. Our research team is is relatively small I think definitely compared to like a lot of the like the isomorphics deep minds.

1:26:55 · Uh like our research team is like you know in the around 10 people. Um, so like we're we're a relatively small team, but we kind of think of it as almost like an investing job where like you're investing ideas towards compute. Uh, in the same sense you're really just capital allocators in that respect.

1:27:10 · Yeah.

1:27:10 · I actually think maybe this is too cute, but I would even make the broader point which I think every we kind of think of everyone at Chai as a bit of a capital allocator. So I think one of the things that surprises people is we're we're pretty small. we're we're only 30 people and that's because everyone we hire onto the research team or the engineering team, you know, especially now that they're in some ways like very empowered with AI, a lot of it is just like allocating, you know, their attention into the right ideas and allocating their compute.

1:27:35 · This is actually I think a characteristic to some extent of machine learning AI projects and also science, right? or whereas if you're building like a API for some B2B SAS company that's not building foundation models whatever your limit is mostly people

1:27:53 · right so you're the resource you're allocating is almost entirely people whereas if you're building hardware you're building uh AI models you're building something scientific then your constraint is those the resources that are you know sort the bottleneck is you know the the lab it's the compute it's other things and So that you have to really be in that mentality of I have these limited allocation of I have some shots on goal. How do I allocate those shots?

1:28:20 · Well, I would say yes and no. So I I agree it's a bit more like that, right?

1:28:24 · But like let's going back to the example of building an API for you know a B2B company, right? That API has incremental cost. You have to support it. It adds complexity to the product. It's another thing you have to go market and sell.

1:28:36 · Maybe you should actually be allocating that into like a different bet, right?

1:28:40 · different thing on your product roadmap that you should be prioritizing instead of the other thing. I think in a world where like building things just gets like really cheap and you know increasingly free. The scarce thing is your the attention both that you can put into it right to keep your product simple and and groable and that your customer can put into it to like really understand how to use it. I see it less as like a a binary thing and more just like we're all kind of as engineers going to be a little bit more like allocators of attention.

1:29:07 · Yeah.

1:29:07 · Which is what executives are.

1:29:08 · We're all just becoming every well I mean like [laughter] um I was listening to a podcast with Satier at Nadella right he says you know Microsoft wants to make everyone a manager of infinite minds right if you like really take that to your extreme like everyone's going to be an executive I mean I certainly feel like an executive and I talk to Claude every day little suite of interns who are all going out and eagerly solving problems you may or may not have actually wanted but they're solving the problems.

1:29:33 · Yeah.

1:29:33 · So, we have two typical questions that we ask that we've already kind of asked one, but I'm gonna ask it again maybe in more directly is if you and you can both answer this. Um, if you could remove a bottleneck from your problem space um by fiat, what would that be?

1:29:53 · That's that's an interesting question. I think one thing that would be really nice like just I'm like always in research land, very hard to turn off.

1:30:00 · For me, it's probably just the validation loop of protein design in general. Uh, so like just being able to say like instantly like, hey, this thing works, this thing doesn't. There's still a bit of walking around in the dark that you're doing. Uh, just so like, you know, you have you have some ways and like I think at Chai, we've taken this like very seriously, but it's probably along the lines of just like validating hypotheses and like, you know, knowing for certain that things work.

1:30:23 · Yeah, that's unsolved problem for sure.

1:30:24 · Unsolved problem. [laughter] Yeah. And would be hugely valuable.

1:30:27 · Hugely valid. Yeah. I'm going to take a much more abstract answer to that which is actually like talent obscurity. I think you know there's a lot of smart people going and working on LLMs. You know there's a lot of people that are working and becoming software engineers for for SAS right but I think just like not that many like smart people go and work on bio. You know I didn't work on bio like in high school cuz I was like oh I could like pick up my computer and program apps but if I want to work on bio I have to like go study and get good grades in school and like maybe get a PhD or whatever right? And you know, maybe that's one reason for it.

1:30:57 · I think another reason is, you know, a lot of this stuff is really obscure, right?

1:31:02 · Like we threw around a lot of big words during this podcast. You can't really visualize the things. It's one of the things we care a lot about at CH is like how do we make the whole thing feel visual on our website and in the product. Um, and you know, part of the reason we're here is like I, you know, I think, you know, more people should realize like you don't need to like have like a super super super specialist bio background to contribute to this like computationally.

1:31:22 · Um and so um you know I think a lot about like talent flows and like where talent goes in the economy and right you know in the '9s everyone was flowing to talent and you know since the 2000s people have been flowing to tech but you know big tech like ate up a lot of the talent you know until you know a few years ago and now maybe like LLMs and the big AI labs are eating up a lot of the good talent but it's like you know how at the meta level like how do you allocate talent better you know

1:31:50 · selfishly I want more talent going into bio I mean we probably want more talent going into manufacturing and physical world things and these other problems that that the US has. But uh yeah, I think communicating that better would be the thing that if I had a megaphone to to talk to everyone, I would I would try to do that.

1:32:05 · Okay.

1:32:05 · So then that leads to the second question which is and maybe the answer is the same, but what is the takeaway one takeaway that you would like to people to have from the episode? Yeah, I mean I think um you know biology has been this somewhat obscure feeling field where you're stumbling around in the dark. You don't know what you're looking at. You're dealing with um non-determinism in your experiments.

1:32:33 · You're having to do a very long and iterative trial and error loop across a very very long amount of time. And um at some point you're crossing that threshold of what you can do computationally when you can get folding models down to being within you know an angstrom right where you can get design models to give you you know uh hit rates you know north of 50% where now you can put them you know in a in a 96 well plate and actually have like 48 uh interesting binders.

1:32:58 · You start to get to the point where now you can declaratively precision engineer what you want rather than betting on you know nature or trial and error to get you there. Um and I think that um look we had the same thing happen in software where you can write code and you can deterministically get an outcome or in electrical engineering where you know you instead of your schematic being drawn out you can put it in cadence design systems and get it on on uh you

1:33:24 · know it made in software right or or CAD for um for mechanical engineering where you can sort of precision engineer your part and get it printed or manufactured.

1:33:34 · Um, you know, the same thing is happening in bio and it's happening very quickly. Yeah. And that really opens the door for a lot of uh really interesting people or maybe it wasn't as scrutable or accessible before, right? Like software engineers like myself, researchers like Matt, you know, obviously we're still going to want the specialists, but um you know the uh the the generalists can often really accelerate the uh the precision engineering happening in the domain.

1:33:58 · Yeah, I think for for me like the biggest takeaway is that the field is actually working. Uh and like like not only does it have commercial traction, but like the research is like actually showing signs of life. Like it's not even just showing signs of life, like the signs of life have been showed.

1:34:12 · We're actually in a place where like the models work. They're delivering value and like there's still tons of really interesting research problems to solve.

1:34:19 · So I think there's a lot more lowhanging fruit in this field than there would be in other fields. And I think the amount of impact that you can have, especially like as a researcher, is just like unmatched in this in this field. For us, we're all very missiondriven. Uh but even if you're not, like it's a lot of fun puzzles to solve. Like there there's like this kind of 3D geometry angle.

1:34:36 · There's like if you like diffusion models, there's like a million problems to solve in that regard. We have this LLM looking trunk in like Chai 1. Uh there's just so much of like core machine learning is touched by these problems. We're still although we've made a ton of progress, there's still a lot to be done. Uh, and I think it's just like one of the most interesting fields to be working in which like while also having some of the largest impact on just like humanity.

1:35:00 · Thank you so much for making a long journey. It's been a great 22minute walk. Yeah. And [laughter and clears throat] you know, we look forward to tracking's progress. Awesome. Thank you guys. Thank you very much.

1:35:18 · [music]