- 4 weeks ago
Hooked on LinkedIn’s Queens? Gotta extend your Wordle streak in the New York Times games app before you start your day? You’re in good company on today’s Vergecast episode. Allison Johnson is joined by Simon Anthony and Mark Goodliffe, world-class puzzle champs and hosts of the delightful Cracking the Cryptic, a YouTube channel where they solve a puzzle on camera every single day.
Category
🤖
TechTranscript
00:00:00Welcome to The VergeCast, the flagship podcast of looking for naked singles.
00:00:05I'm your host, Allison Johnson, and that makes sense in this context, I promise. It is a Sudoku
00:00:10thing. And on today's episode, I'm talking to a couple of world champion puzzle solvers who
00:00:16started their own YouTube channel where they solve a Sudoku every day. They're ridiculously hard and
00:00:22surprisingly fun to watch. After that, I'm talking to Mark Lavoie, who's one of the pioneers of
00:00:29computational photography and really just phone camera technology in general. We're going to talk
00:00:34about his new camera app called Project Indigo. That's all coming up right after the break.
00:00:44All right, we're back. I am joined by two very special guests, Simon Antony. Hi, Simon.
00:00:52Hello.
00:00:53And Mark Goodliff. Hey, Mark.
00:00:55Hi there.
00:00:56They are puzzle champions, but more importantly, the hosts of a YouTube channel called Cracking
00:01:04the Cryptic. And I guess I'll let you guys explain what that is, what you do there. Simon,
00:01:11maybe can you give our listeners a quick rundown if they're not familiar?
00:01:15I can, but it'll sound mad.
00:01:19We love that.
00:01:20Cracking the Cryptic is a YouTube channel where we solve variant Sudoku puzzles sort of live
00:01:30on camera a couple of times each day. And the channel has been going now for about eight
00:01:36years. And it's become like really quite popular. So we have about 650,000 subscribers now. And
00:01:49the average length of a video is perhaps 45 minutes to an hour. So people are spending time with us
00:01:57having a go at these puzzles and then watching us attempt them. And because it's live, it's all of
00:02:06the mistakes, you know, all of the poor thinking is there. But in the end, hopefully we solve the
00:02:12puzzle. And that's what we do.
00:02:16And we each post a video every day. And in addition to that, there's a weekly Cryptic Crossword
00:02:23Masterclass as well. And we solve Wordle every day as well. So there's a lot of content and it keeps
00:02:29us very busy.
00:02:31I really appreciate the Wordle videos. They make me feel, they usually make me feel better.
00:02:37I'm like, okay, this wasn't as easy as I thought it was. And if you could explain really quickly.
00:02:45So I discovered your channel because I got into Sudoku. But I am more familiar with,
00:02:53I think what you call classic Sudoku. Could you explain just the basics of classic Sudoku if
00:03:03people are not familiar, but also what, how it escalates from there and the kind of puzzles
00:03:08that you guys tackle?
00:03:09Sure thing. So in normal Sudoku, you get a grid of nine by nine cells. And you have to put the
00:03:18digits one to nine once each in every row, every column, and then there'll be nine three by three
00:03:24boxes in the old nine by nine grid. And each of those three by three boxes mustn't have a repeated
00:03:29digit either. And that's the game that I think most of us were familiar with when it first came to
00:03:36the West, I suppose, which was in early 2000s. And variant Sudoku takes that and sort of turns it up
00:03:47to 11. So you might have extra graphics in the grid, there might be a thermometer wending its way
00:03:54through the grid. And you'll have to increase the digits along the thermometer from the bulb end.
00:04:01And then it can get crazier and crazier from there. We've got puzzles now where there's a rat in the
00:04:08grid that has to get to a cupcake going along certain pathways. We've got puzzles which have
00:04:15where most of the grid is covered in fog and you can't see it. But when you get a correct digit,
00:04:21some of the fog will clear. So the puzzle becomes, is revealed to you gradually. And these puzzles have
00:04:29all been handmade. So they're very, very different to most people's experience of Sudoku, which will be
00:04:36on the back of newspapers. All of those puzzles are made by computers. Whereas the moment you solve
00:04:45a puzzle that's been created by one of these fantastic human brains, it's a different experience
00:04:54all together. And I think that's why the channel has become popular, is people are suddenly seeing
00:05:00just the art that can be found in the puzzle form.
00:05:08I find that so interesting because maybe one of the things I do when I get into a thing is be like,
00:05:15how is this made? I want to take it apart, you know? So maybe if you could talk a little bit about
00:05:22how you mentioned a lot of Sudoku, classic Sudokus are computer generated. Were they always like
00:05:29that? Is this like a recent thing? And how can you tell the difference, you know?
00:05:35Well, I mean, originally the puzzle was invented in Japan by a company there called Nikoli,
00:05:43which is a wonderful place of innovation. And all of their puzzles are handmade. But when the puzzle
00:05:48came to the West, it was Wayne Gould originally, I think, a New Zealander who invented a computer
00:05:54program that could just churn out zillions and zillions of puzzles. And he managed to sell it to
00:05:59many of the newspapers in the Western world. And so it just became accepted that, you know, you could
00:06:06have computer puzzles and everyone seemed fine with that because they'd never experienced anything
00:06:12different. Whereas when you solve a Nikoli classic Sudoku, so the classic Sudokus that you were
00:06:19originally used to, Alison, you know, when you solve one of these Nikoli puzzles, it's completely
00:06:25different because the constructor has built into the puzzle sort of a trail of breadcrumbs. And you can
00:06:34feel what they want you to find at the particular points. And there's normally some really beautiful
00:06:39logic packed in there, which you just don't find in the machine generator puzzles. So
00:06:46yeah, that's, it's a big difference. Yeah. When we started the channel, we were mainly solving
00:06:54classic Sudokus from, say, the New York Times or something. And they would all be computer generated.
00:07:01And they would often have a very standard kind of trick to get them done. And we would go through
00:07:07that. But then gradually, as we got going, we, we wondered, would it be interesting to show people
00:07:13the sort of variants Sudokus from world championships that we'd experienced that were a bit more
00:07:18interesting in a way. And unbelievably, this whole community grew up watching these puzzles,
00:07:24watching these videos and going, Oh no, that's, that's really interesting Sudoku. And I think almost
00:07:31all the setters that we feature on the channel now, and we, we basically take submissions from people
00:07:37as, as a good puzzle to do on the channel. Almost all of them grew up, learned about variants Sudoku
00:07:43from our channel. So it's, it's marvelously organic in a way. Yeah. I was wondering if you could talk a
00:07:50little bit about the community, because I think there, there's an interesting moment, you know, with,
00:07:57with any kind of creator on the web, you know, whether you run a website or a channel where we're sort of
00:08:04discovering just the real value to having a community and, and that's kind of what people
00:08:10want. So how do people connect with you? You, you mentioned they suggest puzzles, but how else are you
00:08:16interacting with your community? Well, it's one of the things that happened to us. You have to bear
00:08:23in mind, we're sort of middle-aged guys. We're not, we're not tapped into things like discord.
00:08:29Uh, when the channel started to get bigger, we got approached by some people who watched it and said
00:08:34that you really should have a discord channel, uh, associated with your, um, with your YouTube
00:08:42channel. And we were like, Oh, that sounds good. And they said, Oh, we'll, we'll take on the sort of
00:08:48role of setting it up for you. And we thought that'll be, that'll be nice. I think there are now nearly
00:08:5340,000 people in this discord server. Um, and all trading ideas, all solving puzzles for each other,
00:09:03trying to push forward the boundaries of what's possible in Sudoku. I mean, I, I shudder to think
00:09:10what the combined IQ is of the people on there, but it is, there is some serious weaponage in terms of,
00:09:17I, you know, intellectual firepower. Um, and one of the lovely things about cracking the cryptic is
00:09:25that actually the community is remarkably kind. Um, it's something we've always encouraged, you know,
00:09:34kind comments, positivity, and people really seem to take that to heart. Um,
00:09:41um, I think most of the internet now is probably a place where blood pressure gets raised, whereas
00:09:47cracking the cryptic is a place where it sort of gets calmed a bit. And, um, it, it, uh, yeah,
00:09:55it's, it's, it's, it's attracted, it's attracted people to it who are in general, very decent and
00:10:03normally very bright as well. So yeah, a lot of imposter syndrome, at least I do.
00:10:09Oh my gosh. I mean, having to solve everything live and kind of be on your feet like that,
00:10:16are, is it, um, is it a challenge every day? Do you get kind of nervous? Um, how does it-
00:10:22It is daunting. It is a worry that you, you sit down in front of a puzzle and if it goes wrong,
00:10:29I mean, if it goes completely wrong and you can't solve it, we don't have to put up a video. So
00:10:34there's always that get out, but that's kind of a rarity, I think for both of us. And, and most of
00:10:40the time, if you do make a mistake, well, you have to kind of backtrack and find it and work out what
00:10:45you did wrong. And people are very generous about that as well. Um, I think as Simon said, the, the,
00:10:51the levels of kindness are very high and, um, and the, the great setters and theory creators
00:10:59seem very keen to still interact with us, even though I suspect we've been revealed as not the,
00:11:05not the great brains that people maybe thought we were at the beginning. But, um, it's, it is just a
00:11:12pleasure to, to deal with almost everybody on the site through discord in the comments. Um,
00:11:18we get emails, we, we put our email address out every day and we get hundreds of emails,
00:11:23um, as I say, puzzle suggestions, but also lots of general comment. I think we were amazed at the
00:11:30beginning when it started to grow and people would tell us something like, Oh, I fall asleep to you
00:11:34every night. And we weren't sure whether that was a compliment or an insult, but, um, gradually we,
00:11:40there was a period and it's probably still ongoing where we get about one email a day about how we've
00:11:47seriously helped someone's mental health to, to, you know, from extreme levels of stress sometimes.
00:11:55And we've had at least three doctors tell us that they would have given up their course,
00:12:00but the calm logic that we allowed them to see enabled them to keep going in their studies. And
00:12:06it's just, it's an amazing thing to have that kind of impact with no, no thought or plan of doing so.
00:12:13Yeah. Yeah. There is, um, kind of strikes me as like solving a puzzle is a pretty solitary
00:12:19activity. I mean, it kind of has to be unless you're doing a jigsaw puzzle with your friends
00:12:25or something. Um, and, and sort of personal in a way, like when I sit down to do Wordle every morning,
00:12:33it's a little, I feel a little embarrassed to like show other people my guesses, you know,
00:12:39you're, you're sort of like, it's sort of like when you talk on the, you're like ordering a pizza on
00:12:44the phone and I don't, I'm like, I don't want to hear, I don't want anyone else to hear me like
00:12:49working this out. So I wonder if there's an element of just, it, it feels like you're exposing something
00:12:57a little bit and people are connecting to that in a way where it resonates, I guess.
00:13:02I think there, there's definitely something to that. I mean, it is, it's, it is nerve wracking
00:13:10to actually solve. And because all of the thinking you're doing is live, I mean, I still get this,
00:13:17um, this shot of adrenaline if I have to do mental arithmetic, because some of the arithmetic we have
00:13:22to do is it's, you know, you could be multiplying some reasonably big, you know, two digit numbers or
00:13:30something like that, but you might be having to find the prime factors of something that's,
00:13:34that's, uh, not trivial. And it's, it's, you're sort of there thinking people are going to be
00:13:42watching me thinking he's so stupid. Why can't he do it more quickly? And you know, you've got this
00:13:49devil on your shoulder the whole time chirping away about how, how you're being dumb and you have
00:13:54to ignore him because he's, he's just a very annoying creature. Um, but he's definitely there.
00:14:01Yeah.
00:14:02We also have different strengths. I think Alison will sometimes I marvel at Simon's instinct for
00:14:09a break in, in a puzzle that's complicated to start. And I just can't see how he's honing in on
00:14:16the right spots as he so often does. Um, he's, I understand jealous of my sort of basic Sudoku
00:14:23mechanics and, and maybe some of that mental arithmetic as well. But, you know, these are
00:14:29the skills that I think are trivial, whereas he has skills that, you know, I can sit in front of
00:14:34a really hard puzzle for 20, 20 minutes, half an hour, 40 minutes, just not knowing what to do.
00:14:40And it just feels dreadful in a way, but you, you know, you learn over time that if you keep
00:14:46worrying away at it, you can, and this is very important to remember that the constructors are
00:14:52setting you a puzzle that they want you to have fun with and struggle with, but they want you to
00:14:57get there. So there is this trail of logic to find. And when you do light on it, you get this
00:15:03wonderful epiphany. And I think that's what keeps people coming back to the channel is watching those
00:15:08moments where you suddenly understand something and that's joyful.
00:15:13No, hang on. Maybe it's, no, it is the same. If that's a six, I can't put a one here. And this is a six
00:15:23in box five, which puts a one here. So you can't put a one either side of it. So here we go. So now
00:15:28I've got, now I know there's a six in this domino, which means this is not a six. This is a six, which
00:15:35means this is a one, this is a two, and we are cooking with gas all of a sudden.
00:15:41Yeah. And I, I kind of see, I don't know if I'm just like keying into this because I've been
00:15:47interested in Sudoku. Um, but there's a lot of like puzzles seem to be having a moment. Like I know
00:15:56a friend of mine is addicted to the puzzles in the LinkedIn app and like opens LinkedIn every day.
00:16:04And I, I'm sure there's an element of they've discovered, you know, they can keep people coming
00:16:10back every day and get those like daily user numbers up. But, um, what do you, what do you
00:16:16think that says about like BBR moment culturally or what puzzles are doing for people?
00:16:21I think there's a, there's a lot to unpack there. I mean, I think LinkedIn has really,
00:16:27it dodges most office firewalls, of course, which means that it's absolutely brilliant. If you, if
00:16:34you're, if you enjoy having a few, few minutes out every day and you can, a lot of puzzle websites
00:16:40tend to be blocked from my experience, but LinkedIn probably won't be. Um, and you know, so that's a good
00:16:47thing, but I also, I sense that there is being a bit geeky or nerdy, which I've been throughout my
00:16:56life. I mean, yeah, I've never, never been cool, popular. I mean, my sister has relished throughout my
00:17:05life, you know, telling me how, you know, useless I am in all social situations. Um, but now it just
00:17:14feels, I don't know whether it's AI or, you know, computer programming becoming, you know, it's,
00:17:21it's everywhere, isn't it? And so necessary for so many parts of life, but it feels like,
00:17:28oh, the popularity of things like escape rooms as well. Um, or maybe it's the mindfulness that
00:17:35comes with doing puzzles. Um, as you've found you sort of, when you, when you actually have to focus
00:17:41on a puzzle, you're taken away from maybe the worries of life for a few minutes, you have to
00:17:47focus on what you're doing or you probably won't get very far and it brings a calmness and maybe
00:17:53people value that in this day and age more than they perhaps would have done 15, 20 years ago.
00:18:01I think the way you expressed it, Alison's absolutely right. Puzzles are having their moment,
00:18:04but in a way, what it is, is I think people are in this moment, discovering puzzles and finding out
00:18:11how, how much fun they are. I come from originally before Sudoku from a world of cryptic crosswords
00:18:17and people have often asked me why, you know, why are you doing, or now, now it's obvious,
00:18:22but in the old days they would ask me, why are you doing this old man's pursuit? And I was saying,
00:18:26well, it's, it's not, doesn't have to be. Basically, once you find the joy of cryptic crosswords,
00:18:32there's no reason ever to stop doing them and people don't. And therefore the average age tends to skew
00:18:37older than for pursuits that people will give up when they've had their fill, but you don't do that.
00:18:43And I think it's true about Sudoku as well. People stay with it. Even if they don't stay with our
00:18:48channel, I think they stay with Sudoku and with puzzles. And it's, you know, we've been part perhaps
00:18:54of leading people to discover puzzles and that's great. And I think it's more that people are sort
00:19:00of free to enjoy them now rather than that they, um, that they hadn't thought of them before.
00:19:06I'm curious actually about kind of the tools you use in your puzzles, because, um, I think maybe
00:19:14Simon, you mentioned in a video that you sort of prefer puzzles that you can just think through.
00:19:22And there are, um, tools that exist for super hard Sudoku that like, um, help you math your
00:19:33way out of it. I don't know. Can you, can you kind of talk me through like, um, the tools you
00:19:38do and don't use when you are solving puzzles?
00:19:42Yeah. I mean, I think we both use compute, obviously, because we're, we're recording something
00:19:49we're doing on a computer. We have some great software that's been created, um, by Sven, uh,
00:19:55Neumann who, um, which is very, very good. It allows us to employ some techniques that are much
00:20:04harder to do if you've just got a pencil. So for example, we can color in cells, um, or we could even
00:20:12draw lines through cells. And when we're solving these harder puzzles, things like that, they're pretty
00:20:19useful. The software even, I mean, we don't use it, but it'll allow you to compute all the different
00:20:26ways of adding up to 25 with four digits. It'll actually give you a list of all the ways that you
00:20:31can do that. So if you, if you don't like doing the arithmetic, you can still, you know, access the
00:20:38puzzle. But I think as, as with anything, what you find is you build up your own personal tool of,
00:20:45I don't know, tricks, techniques, and you cycle through those. Um, and that, that only really
00:20:52comes with experience. And I guess now Mark and I have a lot of experience. So we've got something
00:21:00like 4,000 videos now, Mark. I was, I think it may be 5,000 now. Okay. 5,000. Oh, wow. Yeah. Yeah.
00:21:10It's, um, it was a back catalog. Yeah. It is. You do build up a lot of experience and it's often
00:21:17very helpful. And yet still every few months, something completely new is kind of forced in
00:21:25upon us by a brilliant constructor. And we learn something totally new about even sometimes about
00:21:30a basic Sudoku grid that it must have some property that we didn't know before. And, um, that, you know,
00:21:37I still think in some ways this is the dawn of Sudoku, which is brilliant. Yeah. Mark's right. I mean,
00:21:43there was a mad thing we discovered the other day for the first time, which is sort of, if you check,
00:21:48imagine that you've checkerboarded your Sudoku, the sort of internal checkerboard you've created,
00:21:56any digit you put in that has to appear an even number of times in that internal checkerboard,
00:22:01which it's like, why, why, why is that true? And yet sort of geometrically, mathematically,
00:22:08it is true, but we, we didn't know it. I didn't know it before I did that puzzle. It was completely,
00:22:13it's like, it's a truth that exists underneath the surface of the puzzle.
00:22:19It's not going to help you solve a classic Sudoku, but it's there and it's just sitting there like
00:22:24a mathematical truth. It's extraordinary. Yeah. Yeah. There, there is something really
00:22:29satisfying, I think, in, in discovering new layers of something you've been enjoying. Um,
00:22:34I do want to touch on, uh, uh, a puzzle you did. Um, and this was, I guess, went viral. Um,
00:22:43we wrote about it, um, on the verge, uh, this was the, the miracle Sudoku. Is it right? And Simon,
00:22:54you, you solved this one. Can you describe what this puzzle was and kind of how it went?
00:23:00Well, I, I can, I mean, but really it was Mark's fault. So what, what happened was we got sent a
00:23:08puzzle, um, by a constructor called Mitchell Lee. Um, and the very, very unusual thing at the time
00:23:17about this puzzle was it looked like a normal Sudoku, except rather than having several digits in the
00:23:22grid, there were only two digits in the grid. Uh, it was a one and a two, and that was it. Um,
00:23:30and Mark sent me an email and said, it would be good if you opened this live on camera.
00:23:36So I took him at his word, set up my machine, loaded it, having started the recording and was
00:23:43faced with this at that time. It just looked, it looked stupid. I thought he'd just done it as a
00:23:48joke. And so I very nearly, thank goodness. I didn't, I very nearly turned off the webcam,
00:23:54um, and sort of called him in high dungeon, but, but no, I could see one thing at the start that I
00:24:02could possibly do. And so I did the thing and then just, it was amazing after, you know, a few
00:24:10more steps, I could see something else. And it was one of those puzzles where you sort of, it was like
00:24:15a snowball, you know, starting an avalanche. All of a sudden you could, this pattern emerged and you
00:24:22could, at the end, I did solve it. Let me just see if I can put that in there. Let's see what we get.
00:24:30Right.
00:24:31It's got to be joking. There is no way that this, well, it might have a unique solution,
00:24:44but it's not going to be findable by a human being. Um, I suspect this is going to be a short
00:24:51video because he is trolling me.
00:24:53I think the reason it was popular was just my sheer astonishment that this could exist,
00:25:04that you could, yeah, that, that somebody had had the idea that somebody had made it.
00:25:10And also that this impossibility was actually solved in about 25 minutes. It was just bizarre.
00:25:17It was bizarre. Exactly my finding when I, when I tried it before Simon had seen it.
00:25:22Mm-hmm. I, yeah. And something, if, if people aren't familiar with the channel,
00:25:27um, you can solve along with these puzzles. So that's exactly what I did, you know,
00:25:34watched you get started with it. And then, yeah, you, you look at it, you're like, there's no way
00:25:40this could be solvable. But I saw you, you know, go through the first kind of steps and, um,
00:25:47tried it on my own. And it, you know, I had to like rewind a couple of times and see what you were
00:25:52doing, but it's so cool to kind of go on that journey with you where I'm like, oh my God,
00:25:58I am solving this. Um, yeah, it just, I had a lot of fun with it.
00:26:04Yeah. I mean, it was, it was remarkable. I mean, we, we got, we got emails from all over the world
00:26:10as a result of that. It was, it was like a moment where, you know, they say everyone,
00:26:15everybody has their 15 or five minutes of fame. It was like that spotlight suddenly turned on us
00:26:21for, for just a, just a scintilla of time. And you sort of got a bit of the glare of a little
00:26:27bit of publicity. And it was, it was very enjoyable because it was so brief and unexpected.
00:26:32Hmm. Yeah. And, and a, a positive, I think, you know, people are discovering it and having like
00:26:40a joyful moment of like, I can't believe I just watched this 25 minute long video, which is,
00:26:47I think interesting. And you kind of touched on this, I think, um, that your, your videos are long,
00:26:54like, you know, comparatively in, in the age of like, certainly TikTok where you watch for,
00:27:02a few seconds and scroll on. Um, can you talk a little bit about how, why you think,
00:27:09why you're able to enjoy, you know, having such a long amount of time to, to show what you're doing
00:27:18and talk about what you're doing? And we thought that people would just want to see the technique
00:27:22would just want the lesson. And then we kind of thought, well, they won't want to watch the rest
00:27:26of the solve. That's just filling in the numbers. And we, we would try things like speeding it up or
00:27:31putting music to it. And they got really annoyed. They said, no, no, no, no. We want to see you
00:27:36solve it till the end. And it is, I think it's what you said earlier that people want to come
00:27:41on the journey of the solve and they want to be part of it all the way. Some people treat it like
00:27:46sport, like a spectator sport. And until, until you put the last digit in the final whistle doesn't
00:27:52blow. So it's, it's still going. And, uh, and, and that extends into the longer videos. I mean,
00:28:00Simon's done videos. Is it three hours you've done now, Simon?
00:28:04No, I've done, I've done four hours, four hours, 10 minutes is the longest puzzle. And you'd think,
00:28:14oh, well that can't be popular because how could, how could it possibly be the case that anybody would
00:28:19want to watch a man solve a Sudoku puzzle for longer than the extended edition of return of
00:28:25the King or something? It cannot be, it cannot be right. And yet, and yet it's one of our most
00:28:32popular videos of recent times. Um, it, it's, it's like they, people get invested in it. And then in
00:28:42fact, on that video, as one of my favorite comments underneath the video that we've ever had,
00:28:49which was from a guy and he said, 15 years ago, I watched the whole of the Twilight series with my
00:28:58girlfriend on the condition that one day she would watch something of my choice without complaint.
00:29:05Tonight's the night. Oh my gosh. Yeah. Personally, I'd rather watch four hours of puzzle solving,
00:29:17but yeah, I can see how that, that trade was made. Incredibly those long videos get more average
00:29:25watch time percentage than the shorter ones. It just defies belief, but people do, do enjoy the,
00:29:32I think they love seeing us challenged. They love the struggle because they're being challenged as
00:29:37well at the same time. And, you know, it's, there's an extent of seeing what the human brain can
00:29:42achieve. And I'm talking about the constructors more than us, but, but there's us as well.
00:29:48Well, thank you so much for joining me. This was a delight and I hope people will check out
00:29:53cracking the cryptic on YouTube. And, um, I'm, I'm going to be seeing you again soon because I'm
00:30:00going to be watching your videos. Thank you. Thanks, Alison. It's been a big pleasure.
00:30:07Yeah, absolutely. Lovely. Thanks, Alison. We need to take another break. When we come back,
00:30:12we're going to be talking to Mark Lavoie about his new camera app project Indigo. It's coming right up.
00:30:21Welcome back. I am joined by someone who for a portion of our audience will not need an introduction,
00:30:27but we'll do one anyway. Um, I'm here with Mark Lavoie. Hey, Mark.
00:30:32Hi. Nice to be with you.
00:30:34Thanks for joining. Uh, Mark Lavoie is a professor emeritus of computer science at Stanford. Um,
00:30:41but right now and what we're going to be talking about, um, he is a VP and fellow at Adobe where he
00:30:48has been working on a new camera app. Um, also known to a lot of people in our audience in particular
00:30:54for his work on Google pixel camera for, for many years. So thank you for joining me.
00:31:00It's great to be here.
00:31:02So I was wondering if you could give us maybe a shortened version of your bio. Um,
00:31:09there's a lot of interesting stuff on there, you know, some of which I just touched on. Um,
00:31:14but then catch us up a little bit, um, on what you've been doing since you've been at Adobe.
00:31:20Uh, shortened version of the bio. Well, so, uh, I, um, seem to have wandered, uh, from topic to topic
00:31:29roughly every 10 years. So in the seventies, I worked on computer assisted cartoon animation
00:31:35at Cornell in the eighties. I worked on medical imaging at a university of North Carolina. And then
00:31:42at Stanford in the, uh, 1990s, I worked on three-dimensional scanning, including the digital
00:31:48Michelangelo project in the two thousands. I worked on light fields and camera arrays, um,
00:31:55built some big camera arrays at Stanford. And in the 2010s, uh, largely at Google,
00:32:00I worked on computational photography, including, as you said, uh, for the pixel phone.
00:32:06Yeah. And you, um, you joined Adobe in 2020. Is that right?
00:32:11Right. During the height of the pandemic.
00:32:13Ah, an interesting time to switch jobs. I did it too.
00:32:16Indeed. Indeed. Yeah. Um, so I think, and, and you spoke to our editor in chief,
00:32:24Nilay Patel, um, around that time and the stated goal was to create a universal camera app. Um,
00:32:34and it's been five years since then you've released that camera app called project Indigo. Can you give
00:32:41people a quick, uh, the quick elevator pitch if they're not familiar with it? I know a lot of our
00:32:47audience is. Uh, depends how long the elevator is. Uh, so the basic idea there is, um, there are a
00:32:54number of limitations of most smartphone camera apps. Uh, I'm certainly familiar with those
00:32:59limitations. Uh, there's a smartphone look that a lot of people don't like that might look okay on the
00:33:06small screen or in bad lighting, but don't look okay. If you look at that same image, large, um,
00:33:11it's very bright. It's a contrasty. The shadows have been raised a lot. Um, the edges are very
00:33:18sharp. The saturation is boosted. It's a certain look, as I said, it might look fine on your screen,
00:33:24but not if you're trying to make something larger out of it, or even look at it on a large screen.
00:33:29Uh, another issue with most smartphone camera apps is they don't offer much in the way of manual
00:33:35control. Some of the third party apps do, but then they don't offer the third thing, which is
00:33:40computational photography. So our computational photography, meaning combining multiple images
00:33:46to make a better quality image, regardless of the look. So we wanted to combine all three of those
00:33:52things, a more natural look, uh, full manual controls and the best computational photography,
00:33:59the state of the art allows. Okay. We also wanted to release some fun stuff on the side. So we have
00:34:06synthetic lung exposure. We have removing window reflections, which also, uh, ships in Lightroom
00:34:11and Camera Raw. Related to that, you call this look kind of HDR-ish, um, which is, I think,
00:34:20a different thing from HDR as a term. Can you, am I right in thinking that? Do you have that same
00:34:27kind of sense of it? Yeah, yeah, yeah. It's a little bit confusing. So HDR just means that there's
00:34:32some way you can display stuff that's really bright. And, um, to some extent, just the hardware
00:34:39of the, of the phones allow this. They are very, very bright. Uh, and certainly, uh, some display
00:34:46screens, computer monitor screens are also very bright. There are these great, uh, ads or reviews
00:34:52where someone puts on a pair of sunglasses before they even look at their screen. So that gives the
00:34:57capability to display HDR, but the real world can be even brighter and not all phones can display,
00:35:06uh, HDR as being really bright. So in either one of those cases, a scene that's really bright
00:35:12or a display that's not so bright, you need to tone map it in a way that fits all of the
00:35:18dynamic range, meaning the range from black to white into whatever the display can do.
00:35:24That usually involves lowering the highlights and raising the shadows. If you do too much of that,
00:35:32that's what people call the HDR-ish look. Can you talk a little bit about like how we ended up here
00:35:37with this look? Well, they want the whole image to be readable and they want people, people to be
00:35:43able to take photographs in very challenging lighting. So they'll take a backlit portrait
00:35:49with a sunset in the background at the beach, but they still want to be able to see the features of
00:35:53the person who is silhouetted. That's a very high dynamic range situation. And, um, they'd like to be
00:36:01able to share it. So they want it to look readable. And so they'll raise the shadows a lot and that can
00:36:08lead to this HDR-ish look in particular, if you're looking at a small screen in bad lighting. So bad
00:36:14lighting might, for example, be a very bright day where there's glare on the phone surface, then they
00:36:20want to make sure it's readable there too. And so they'll raise the shadows and lower highlights.
00:36:25But if you take that same image and look at it in a slightly darkened room or on a larger screen,
00:36:32then that's where it begins to look, as you said, grayish or, or too squished toward the middle.
00:36:38In the past few years, maybe it's been kind of brewing longer than this. Um, you know, the ultra HDR
00:36:45format has kind of come to prominence, um, which is related and sort of in the mix here, but is a
00:36:54different thing than, than the two things we were just talking about. Um, can you explain what,
00:36:59what that is? I know, um, you know, Android supports ultra HDR. Apple has a version they support, um,
00:37:08just what is this at, at the basic? So both of those are closely related to, and were co-developed
00:37:15with Adobe's format for, uh, representing high dynamic range images. Uh, as a matter of fact, uh,
00:37:23Apple's Paul Hubel and, uh, Adobe's Eric Chan presented the format together at a conference.
00:37:31Okay. Um, so it's really a co-development. Um, ultra HDR is just a brand name. Uh, the basic idea
00:37:38is that a file ought to contain two images, a base image, which is, uh, what you would display if you
00:37:47only had a standard dynamic range display. In other words, not a super bright display SDR for short
00:37:54and a separate image, which is what's often called a gain map, meaning how much would I add to that
00:38:02if I had a brighter display? And that's the highlights part of it. Now it's a little bit
00:38:08more complicated than that. It can be done as either an addition or a subtraction, but that's kind of the
00:38:12basic idea that there are these two images. And if you have an HDR display, then the idea is that
00:38:18the viewer, meaning, for example, the browser software would have the two together and then
00:38:23display that. So that's what all of these HDR formats have. There are other HDR formats. AVIF has
00:38:31one, but the world seems to be moving gradually toward either the ultra HDR or Apple's HEK where, uh,
00:38:38HEK format or ours, which is this JPEG that has a base image, which is SDR and a base plus gain,
00:38:47which then a gain map, which gives you HDR. Why does this one seem to be the one we're landing on?
00:38:53And does it, does it solve our problems with HDR and HDR displays and HDR-ish photos?
00:39:02It's arguably better than the other ones because it really kind of separates out. Here's what you would
00:39:06see if you had it only an SDR display. And here's what you would see if you had an HDR display.
00:39:11And it also means that in image editors like Lightroom, you could separately tune the two looks
00:39:19because you might want a different crushing of shadows or lowering of highlights, uh, for standard
00:39:25dynamic range display and for high dynamic range display. Or you might just want an SDR image and
00:39:31you never want the highlights to be bright. And so in Lightroom, again, you could toggle off the HDR
00:39:37and just sit there and adjust for SDR. So it's a, it's a good format. I don't know if it's the best
00:39:44ever, but it's a good format. And it's nice to see that it's being gradually, uh, adopted. Gradually,
00:39:51of course, is very gradually in this fragmented world of ours. So social media apps are only slowly
00:39:58coming on board with this format. Yeah. There's so many components to it. There's the, you know,
00:40:06the camera, there's the display on the camera, there's where the photo eventually ends up,
00:40:12the display of the person who's looking at that photo on Instagram. Um, and I think something I've
00:40:19seen in me, I, you can let me know if, if you've seen this too or not, but I'm seeing kind of an
00:40:26association from, from photography, kind of like literate people where they, they have just gotten
00:40:34a bad taste in their mouth with HDR and they sort of say HDR is a bit like a dirty word. Um, and
00:40:42I'm trying to evangelize the ultra HDR a little bit, you know, like this is a different thing.
00:40:48Do you see that conflation happening? Absolutely. Uh, I think you're, you're right. But what they're
00:40:55conflating is the ability to display HDR and the particular tone mapping and look that has been
00:41:03used on HDR capable photographs. And that's this over tone mapped thing. And so, um, I think it's
00:41:11possible to display HDR images in a way that doesn't have this look. Interestingly, it's also
00:41:18possible to overdo the look on purpose in a way that's actually artistic. Um, and one person I
00:41:25think of there is Trey Ratcliffe in Stuck in Customs. He's got this great image, one of his
00:41:31signature images of, um, the landscape around Guilin in China that is ultra wide angle and very
00:41:39over tone mapped on purpose and made saturated in a way that makes it look like a fairy tale landscape.
00:41:44So that's art. It's obviously not the way it looked, but everyone knows that when they look
00:41:51at it. So I'm separating out this art, uh, from it. Um, but for ordinary photography, I think you're
00:41:58right. But the answer is that it's, it's in the tone mapping. There's nothing inherently wrong with
00:42:03the HDR capability. Okay. Um, well, I'm telling everyone that Mark LaVoy says I'm right. Um, and I
00:42:10will take that to my grave. Um, you know, the one funny thing about this is, um, there were videos
00:42:17made while I was at Google about, Oh, uh, the look of pixel phones is learning from Italian art.
00:42:23Um, and I talked about my preference for Caravaggio and for, um, Titian and so on and so on. Well,
00:42:31those are all standard dynamic range images. They're just reflective. They're not luminous.
00:42:36They can't do anything brighter than the white of the gallery wall. And so we sort of do need
00:42:43different models and artistic models and a different artistic exploration as we start to
00:42:49do high dynamic range. And I think the world is still experimenting with what it ought to look like.
00:42:56Yeah. How far is too far? What do, what do we all like and dislike? Yeah. Um, how did you think
00:43:03about that going into developing your camera app? You know, I've used the app certainly quite a bit.
00:43:09I know a lot of our readers have, but the kinds of things that people can expect, maybe as far as
00:43:15HDR processing when they use Project Indigo. Right. So I led a team of people who, uh, determined the
00:43:24taste for the early pixel cameras. I don't know if I would go so far as to say I was the tastemaker,
00:43:29but I was one of them. Uh, here at Adobe, uh, I've let Florian Kynes who co-wrote the, uh, the blog
00:43:37with me be the lead tastemaker. And you can see his taste in, in all the photographs that are on the
00:43:43blog. They look natural. They look like, oh, if I were there, I know what I would have seen.
00:43:49They don't look like there's been some deliberate adjustment and tone mapping. They look like if I had
00:43:54an SLR with enough dynamic range, that might've been what the picture looked like. Um, and so one
00:44:01of the characteristics is let dark shadows be dark. And I think a lot of people just like that.
00:44:10Yeah, I know I do. Um, there's, to be honest, we've kind of fell into this moment of a trend toward
00:44:18retro and toward film looks, which are more natural looks because there's no, um, adjustable
00:44:26tone mapping or there's very, there are fewer ways of doing adjustable tone mapping in film
00:44:32and people like that. And they sort of heave a sigh of relief, like, okay, you know, that looks
00:44:38like a plausible scene and the darks are dark and we like dark because it's the et cetera principle.
00:44:46You know, what's the mystery in that dark area there? And I like that and Florian likes that.
00:44:54And, um, so that's kind of what we allowed to happen on, on Indigo. Yeah. I should, I should
00:45:00point out though, that, that it's a profile. If you load it into, into a Lightroom or Camera Raw,
00:45:07it shows up, says Indigo and you can change it. It's a, uh, in particular, if you've been
00:45:15capturing DNGs, which means raw images, you can change it to any profile you want. You can add
00:45:21presets to it. You can make your own look. So it's not like we're demanding that this is the look,
00:45:27um, that you would produce with that app. It's our suggested look, but you can change it.
00:45:33Yeah. Um, in kind of along the same lines, I wonder if you can talk about the raw capabilities.
00:45:38I'm someone who kind of poo-poos shooting raw on a smartphone. I'm tend to want to let,
00:45:46let it just do its thing. And that might be, um, maybe, maybe a older way of thinking about
00:45:52smartphone photography, especially with computational raw, which you've brought into
00:45:58project Indigo. Could you explain, uh, quickly what that is and why that was important to have?
00:46:05Right. So if you do JPEG only with Indigo, you'll get our look. If you bring it into
00:46:13an image editor, uh, Lightroom or any other editor, you can make minor tweaks on it, but you can't do
00:46:20anything really major because the characteristic of any of these, uh, tone mapping methods is that
00:46:26the highlights in one part of the image might be the same pixel value as the shadows in another
00:46:34part of the image. And so if you use sliders or tone curves or anything to raise one or lower one,
00:46:42you might raise or lower other things you didn't want to lower or raise, even though they were
00:46:46actually different brightnesses in the original scene, they got tone mapped to be the same pixel
00:46:51value. So this is the main reason why people like shooting raw because the, uh, the numbers that
00:47:00are in the file represent relative scene brightnesses just as sensed. And then you can keep the shadows
00:47:09separate from the highlights if you wanted to. So that's the main reason for shooting raw. Now you
00:47:15say you don't really like, or have not really shot raw on smartphones. So the reason for that is that
00:47:21the raws have not been very good. They both don't come with a very good look and they often don't come
00:47:27with computational photography or enough computational photography. So in low light, they look noisy on
00:47:34some phones, which I won't name. If you switched into raw mode, it would take a single frame and end
00:47:42up being very noisy because the smartphone sensors are small. So the right thing is do this computational
00:47:49photography where you align and combine multiple frames, even to produce the raw. Now, some purists would
00:47:57say, well, that's not raw anymore because it was captured over a number of frames. That's true,
00:48:02but it still has the major characteristic of raw that it's proportional to scene brightness. And you
00:48:08can do any editing you want without worrying that you're adjusting both shadows and highlights at the
00:48:13same time. Yeah. So I do recommend that you try raw in Indigo. I think you'll like it. Yeah. Yeah.
00:48:20And I've made an exception for Indigo. Definitely. Just, just knowing, uh, knowing I'm not throwing
00:48:26away a bunch of extra frames and extra data. Um, yeah, it's, uh, it's, it's been a totally different
00:48:33experience shooting with the camera. Honestly, I, with a phone, I, I use the phone cameras on every
00:48:39major phone, you know, released in this country. Um, and this one, this app really made me think about
00:48:47it differently. Um, so. One characteristic, I mean, there, there are trade-offs, you know,
00:48:52why didn't everyone else do the same thing we did? Well, we're capturing more frames. It takes
00:48:57longer to put them together. You have to be a bit more patient between shots, particularly in low
00:49:03light. So there are trade-offs to be made. And we made a particular trade-off favoring the
00:49:09more serious photographer, the one who cared more about image quality and adjustment after the fact.
00:49:16Mm-hmm. Yeah. It, it sort of speaks, I think someone who really wants to get into the nuts
00:49:22and bolts of photography, it sort of speaks that love language where just right from the
00:49:28onboarding screens, you know, you, you learn a lot about what the camera's doing and how things will
00:49:35be processed. Um, and, and with your background as a teacher, um, I wonder how that did this feel
00:49:45like a natural kind of fit? Did you find as you were developing this app, you, you were sort of
00:49:50bringing that part of your background along too? Yeah. I love teaching. Uh, maybe I sometimes talk
00:49:57too much, but I want people to understand. And so, uh, the last course I taught, um, before I, uh,
00:50:05left Stanford first for Google and then for Adobe was a course on digital photography. And I would teach
00:50:12both the art and the science, you know, why do we want to combine images? Where does this noise come
00:50:18from? Uh, what are aberrations on lenses and things like that? And so that kind of spilled over onto the
00:50:26explanations that I made both in the blog and on the opening screens in Indigo. I want people to
00:50:31understand. Mm-hmm. How do you think about that? Maybe line is the wrong word, like a line between
00:50:38the art and the science. I think in photography, especially, um, people can get kind of,
00:50:46they'll, they'll want to dive into one or the other and they'll see, you know, they want to know
00:50:51everything about how the, the camera's working and they get into the, the nuts and bolts or they don't
00:50:57want to get into all that. And they feel like it detracts from enjoying the art. How, how do you see
00:51:03that? Are those two opposing things? Do they, can you exist in two different states at once?
00:51:12I don't consider them opposing at all. I do understand tastes that some people are just
00:51:17nerds about the science and some people are nerds about the art. Um, one of my heroes is Ansel Adams.
00:51:24So Ansel Adams, first of all, was a teacher in addition to being a great artist,
00:51:29but he also embraced both the art and the science of photography. He wrote three amazing books,
00:51:36the camera, the negative and the print, which are quite scientific and full of careful illustrations
00:51:43of how perspective works and the zone system for exposure and so on. Uh, I had a chance once to ask
00:51:51one of his assistants, his longtime assistants, would he have embraced digital photography? Uh, he did have
00:51:58a chance to test a very early digital camera, but would he, would he have embraced it for his
00:52:03photography? And without a moment's hesitation, the, the assistant said, of course he would have
00:52:08just look at his work and his books. Yeah, it, it makes sense. And I think a lot of photography
00:52:16conversations go back to Ansel Adams sort of inevitably when you talk about processing Photoshop,
00:52:23you know, um, maybe even the moment we're in right now with, um, generative AI there, you,
00:52:31you kind of can trace back to like, well, we think of photography as this, you know, you're just,
00:52:38it's just photons hitting, uh, a piece of film or a sensor. Um, but the way Ansel Adams, you know,
00:52:45manipulated images and, um, the things he did in the dark room, uh, people are maybe not aware of,
00:52:54um, exactly. And not only that, so, so first of all, you're exactly right about the manipulations
00:52:59in the dark room, his dodging and burning, which, uh, many photographers know about is local tone
00:53:06mapping. It's the same as this raising the shadows in one part of the image and lowering the highlights
00:53:11in another part of the image, but maybe not in all parts of the image. That's just dodging and burning.
00:53:15Mm-hmm. Um, but more than that, he was open about it and taught about it. He has one great book
00:53:22that is, um, the making of 40 photographs, I think it's called, and he shows the original
00:53:27and the version after he has fiddled with it in the dark room or interpret it in the dark room.
00:53:34Mm-hmm. In fact, one of his most famous pictures,
00:53:36The Clearing Winter Storm in Yosemite is a great example that I used to show in my photography course
00:53:41Stanford of the original, which is surprisingly, I wouldn't say bland, but fairly even toned and not
00:53:52nearly as exciting as his version after dodging and burning. And he would explain what he did and why.
00:53:57It's such a joy to hear someone who is so, you know, immersed in what they do and has such a love
00:54:04for it and the, the technical mastery, I think. So yeah, I definitely need to, to check more of that
00:54:11out. Um, I have some other questions about Project Indigo before we stray too far into philosophy of
00:54:20photography and art. Um, I'm, I'm wondering your thoughts on, um, generative AI and, uh, particularly
00:54:34it's on my mind because I've just been using the pixel 10 pro. Um, and there is, um, a zoom feature on
00:54:44the camera that instead of using, um, um, just digital zoom, it, which is, has a lot of limitations,
00:54:52um, takes your image and, uh, uses a diffusion generative AI model to kind of fill in, fill in
00:55:01the gaps. And as, um, your former colleague, Isaac Reynolds said, squash the artifacts. Um, so
00:55:08that's sort of out there now, is that something you would consider bringing into Project Indigo?
00:55:18Uh, a lot to unpack there. So, um, uh, first of all, in general, generative AI definitely has a place
00:55:28in the future of art, the future of photography. Uh, the gen remove feature in Lightroom is generative AI
00:55:35and I use it to remove distractors in many of my personal photographs. It's very useful. Uh, of
00:55:43course it can be used to make entirely new stuff and there's a place for that as well. Um, so as far
00:55:51as digital zooming goes, um, I don't want to talk specifically about pixel, but there are other
00:55:58smartphone vendors who have explored extreme zooming like that. Uh, any smartphone that does say 100x is,
00:56:07of course, using digital zooming of one digital scaling of one kind or another. Uh, so, um, maybe I can
00:56:16answer it by giving a general way to think about this. Uh, if they are claiming 10x zoom over the, uh,
00:56:26over what the lens can do or 20x over what the lens can do, if it is a native resolution,
00:56:32that's a 5x lens. Uh, but let's take the, the 10x case. Then the way to think about it is every
00:56:3910th pixel horizontally and vertically is real, which means that, uh, considering both horizontal
00:56:48and vertical, um, uh, one out of 100 pixels is real. The other 99 have to be made up. Now,
00:56:59how do you make up pixels? You could just interpolate smoothly and make a blurry image.
00:57:06Um, you could try to sharpen edges. Now, why would you sharpen edges? Uh, because you have some prior
00:57:15knowledge that the world is composed of mostly smooth areas and sharp edges. That's prior
00:57:22knowledge. That's a buzzword in artificial intelligence. If you have a large training
00:57:27set, you have more prior knowledge about the world. And these diffusion models have a lot
00:57:34of training data and therefore a lot of prior knowledge. So they'll look at every 10th pixel
00:57:40or every hundredth pixel in area and say, oh, that looks like a building. Buildings have windows.
00:57:48The windows have sharp edges. I will sharpen all that up and make it a stucco facade and then a dark
00:57:55window. And so they're clearly making stuff up. Um, when you're using a generative AI, I think they
00:58:03admit, uh, all of the smartphone vendors will admit that they're making stuff up. The question is,
00:58:09is it useful? Um, what are you doing with the image? Um, and that determines whether it's okay to do
00:58:18that kind of stuff. Um, one thing I will, uh, say more about pixel is they do say they're not doing
00:58:25it for people. And that is an admission that it does matter what the gen AI is used for. Um,
00:58:32but again, back to the general argument, it's, um, useful for some things and not useful or even
00:58:41misleading for other things. And so I think that'll be the discussion that everyone has going forward
00:58:46about the use of gen AI and digital scaling is what's it being used for and does it work or does
00:58:52it make up stuff that's objectionable? Yeah. Yeah. I think there's maybe a parallel to the,
00:58:59the moment we're in with AI generally. And, and I, my framework is what is the features on the phone
00:59:07mostly. Um, and there's sort of a rush to have a bunch of AI stuff and sort of flashy, but not that
00:59:15useful. Um, and I, I think, I hope we're getting to a place with it where we can think of it more as
00:59:22a tool that is just sort of there when we need it and is helpful. And then we don't think about it
00:59:28when we, when we don't need it. It sounds like that's kind of what you're, what you're hinting
00:59:33on to. Right. Let me make one further distinction. There's AI and there's gen AI. Now those terms are
00:59:38kind of fuzzy themselves. Um, but when we think of text to image or text to video, everyone thinks
00:59:45of a certain kind of gen AI and let's use that definition. But AI is much more general than that.
00:59:51AI can just help you answer any question where you might benefit from prior knowledge. So the,
00:59:58um, 10 X multi-frame super resolution that Indigo has is using an AI. It's not using an AI to,
01:00:07uh, to propose what the pixels ought to be, but to help align the multiple frames. The multiple frames
01:00:14all look a little bit different because when you hold your phone, you naturally shake a little bit.
01:00:18And so it gets different views of the world. And that actually provides additional real information
01:00:24about the scene and the AI helps us align it and predict, uh, how we should weight those different
01:00:30images. So that's an AI. It's being used as a tool, but it's not gen AI. And so one more thing I'll say
01:00:38about that is, well, why can't you just keep doing that and take more and more frames and get a hundred X
01:00:43and you can't. And the reason you can't is because there is a fundamental limit on how much detail
01:00:49gets through a lens. It's called the diffraction limit. And every lens has a diffraction limit.
01:00:56And for most smartphones, there might be two X more information in each of horizontal and vertical
01:01:04than the pixels normally allow. There might not, depending on the lens, there probably isn't 10 X.
01:01:11I mean, there isn't 10 X more in each of X and Y for each horizontal and vertical. So at that point,
01:01:18you definitely are making stuff up.
01:01:20Yeah. Right. That, that sort of relates to something I think about a lot with smartphone cameras is
01:01:27the, the role of hardware versus software. It kind of seems like, especially in the, the major,
01:01:35um, players in the U S that we've sort of leveled out on the hardware. You know,
01:01:40we're not talking about a new bigger camera sensor every single year, you know, it's,
01:01:47it's sort of like the same sensor with some tweaks to the software that's, which is not quite the case.
01:01:53I think if you look outside the U S, um, um, I'm my colleague, Dominic Preston gets to play with the,
01:02:02you know, the Xiaomi phones that come out. Um, and they seem to be trying some really interesting
01:02:09stuff with the hardware, the really big one inch type sensors. Um, you were going back to
01:02:16attachments that you put on your smartphone that turn it into more of a camera.
01:02:20Yeah. I assume you realize that the major limitation there is the thickness of the phone.
01:02:25That's what it's all about. Yeah.
01:02:26The, the fundamental optics of a camera means that if you, uh, says that if you want a larger sensor,
01:02:32you've got to have more space between the lens and the sensor. And you can do a, quite a variety
01:02:39of hardware tricks to, uh, finesse that a little bit, but not a lot. And so basically a larger sensor
01:02:48means a thicker phone. And it, yeah, maybe our, our sensibilities are that we, we just don't want
01:02:55that. And we're going to kind of settle on, um, you know, we get what we get. Do you think there is
01:03:01innovation to be had in the camera hardware, you know, as within the bounds of what we'll tolerate
01:03:08as, uh, you know, how thick a phone can be, how big a camera bump can be.
01:03:13So far, all that, uh, innovation has been incremental, but very useful. Um, as a specific
01:03:20example, uh, when I was working on pixel, we used Sony sensors and every year the Sony sensors would
01:03:25get better, just a little bit better, but they really would get better. One of the ways they got
01:03:30better is that what's called the read noise, the extra random stuff that gets introduced when the
01:03:36pixels are read off of the sensor would get less and less. And so in the last couple of years that
01:03:44I was at Google, we could do astrophotography where it's very, very dark and you need that
01:03:49read noise to be small. We can successfully do it. Whereas a few years before we really couldn't.
01:03:56And so these incremental improvements are useful on the telephoto lenses. Uh, many of the smartphone
01:04:03vendors, uh, including iPhone have gone to what's called a periscope or, uh, uh, a bended optics.
01:04:10Yeah. Um, that's an interesting development. So there are hardware developments to be had.
01:04:15They're slow, they're incremental, but they're useful.
01:04:19Yeah. Okay. Well, that, that gives me hope. I don't know. There's something fun about new hardware
01:04:24as much as, you know, software is important. Um, on the subject of, of innovations and things coming
01:04:34up. I wonder if you can give us a taste of what you're working on next for Project Indigo. I know
01:04:39it it's, um, would you call it a beta? It's in the Adobe labs kind of, um, under development state,
01:04:47if that's correct. Right. I wouldn't, uh, the blog doesn't even characterize it as beta. It says
01:04:53experimental. Okay. Yeah. Meaning this might or might not turn into a product for Adobe. Yeah.
01:05:00Um, so in the blog, we talk about a few things. One of them is I would love to do exposure
01:05:05bracketing where you, uh, there are some scenes that are so high dynamic range that just under
01:05:12exposing and, uh, to, to reduce clipping of highlights and averaging of more frames to reduce
01:05:19noise in the shadows isn't enough. Um, the example that we have in the blog is a moon over a moonlit
01:05:26landscape. So the full moon, uh, over a moonlit landscape, the difference in brightness between
01:05:33those two is 19 F stops, meaning two to the 19th power, meaning about half a million to one that's
01:05:42beyond the dynamic range of any camera, even an SLR. So if you wanted to capture a full moon and
01:05:49moonlit landscape, you need to bracket by bracketing. I mean, capturing multiple images with different
01:05:55shutter speeds. And so of course SLRs let you do that. Um, there's some of that in the smartphone
01:06:02industry. We would like to do a lot more of it in Indigo. Yeah. Okay. Another similar thing that is
01:06:09also mentioned in the blog is focus bracketing, meaning let's move the lens in between captures
01:06:17and then combine them to make, um, what is called an all in focus image. So you would think, well,
01:06:24you know, in a smartphone, everything's in focus more or less. Well, that's true for things that
01:06:29are far away, but not for things that are close. So I don't know what your killer app for all in
01:06:35focus might be, but I know my killer app is food, my dinner at a nice restaurant where it's worth
01:06:41photographing. It's very hard to get the whole dinner plate in focus. You can do it if you shoot
01:06:47from above looking down, but not if you shoot obliquely. Yeah. You're so close and yeah, you're
01:06:53like, is the tomato on the top of the salad or the plate behind it going to be in focus? Yeah.
01:06:59Um, so those are two things that are kind of obvious. There are a number of other things that,
01:07:06that we're exploring. Um, they basically fall into the same camp of let's take a bit more time
01:07:13during capture, try a number of different things, combine them computationally. What I'd love to do,
01:07:18as the blog says, is to keep all the data, not just one image so that you could also play with it
01:07:25later in an editor. Yeah. And that's, that's a great example of where the camera app and the
01:07:32editing app can work in synergy. Yeah. A little bit of a, of a nice advantage when you work for Adobe.
01:07:39Exactly. Um, okay. Well, I have to ask for my Android people. Is there an Android version come in?
01:07:46Um, it's definitely on the want list. Okay. Um, I, uh, I used to work on building Android camera apps,
01:07:57of course, right at Google. As a matter of fact, the API, the application programming interface
01:08:02for the camera, uh, was something that I and my graduate students at Stanford designed the so-called
01:08:08camera to, um, programming interface for Android is, uh, came out of a, uh, a project at, uh, Stanford
01:08:17called the Franken camera project. Oh, nice. And it's a great interface and I would love to address
01:08:23it, but Android has challenges. It's a very diverse and fragmented ecosystem. Not every vendor
01:08:33implements the camera to API well, and we'll have to deal with all those challenges if we
01:08:39try to build an Android version of it. Yeah. Okay. That sort of comes to mind. If this is a
01:08:45universal camera app, like, well, I want it on my Android phones. Yeah. I guess I set myself up for
01:08:52this by calling it universal five years ago, didn't I? You did do this to yourself. So, well, um,
01:08:59it was deliberate. Yeah. Yeah. Think big, you know. Um, well, I appreciate this so much. Thank
01:09:05you, Mark. And I know, uh, a ton of Verge readers are interested in your work and we're, we're loving
01:09:13playing with Project Indigo. So they're going to be tuned into, to what you're working on next.
01:09:18Good. And for all the Redditors out there who are one, who are largely positive, which I'm relieved
01:09:25to see. Reddit is not always positive. Yeah. But the one complaint they have is where are the updates?
01:09:33To them, I'll say they're coming. Okay. There you go. Well, thank you so much.
01:09:41It's been a pleasure talking to you, Alison. So we've got to take a quick break. And right
01:09:45after that, we'll be back with a hotline question. All right, we're back. We're going to take a hotline
01:09:56question now. And as always, the number for that is 866-VERGE11. And our email is vergecast
01:10:03at the verge.com. We love the questions, keep them coming. And we answer at least one on every show.
01:10:10So definitely write in or call in. This week's question comes from someone who isn't that concerned
01:10:17about image quality on their phone, but they're a little bit bothered by the camera bump. So we can
01:10:22take a listen. Hello, I am Osvaldo. I'm calling to talk about cameras on phones. I, for one, don't care
01:10:32much about the quality of the camera, to be honest. And I seem that if I want to get the best phone
01:10:42available on the market, I have to basically live with a bump that is almost getting twice,
01:10:49almost to the size of the overall phone. So I just wonder about where are we going with this?
01:10:55Because it seems that these camera bumps just keep getting bigger and bigger. And somehow the phones
01:11:00get thinner on the other side, but then the size of the bump is not taken into account from the
01:11:06overall size of the phone when things get pretty unmeasured. So that seems pretty weird that, let's say
01:11:13that the phone size is an overall five or whatever, and then the bump is at two on top of that, but then
01:11:19somehow it gets reported that the size of the phone is just the lower number. So I don't know. I'm just
01:11:26wondering if there's like any hope out there for a top of the line phone to just basically start
01:11:32moving away from the bump and then having the, having be the case that we can have a phone that
01:11:39has the specs of a top of the line, but not the oversized bump that would allow us to at least
01:11:45have the phone sit like fresh on the table. So I guess that's just food for thought. Thank you.
01:11:51So I need to start with mostly bad news. Osvaldo, I am so sorry. I don't think the camera bumps are
01:12:00going away, but I love this question because A, I appreciate the honesty, really don't care about
01:12:06the image quality, just tired of living with a big camera bump. And B, I think it's just saying out
01:12:12loud what a lot of us are thinking or bothered by. But I know I, for one, just want all the best
01:12:21image quality I can get. So it's sort of something I, I've shoved into the back of my head that I can
01:12:26live with. But I think there's a few reasons why the camera bump is not going away, especially on the
01:12:33pro in the higher tier models that Osvaldo was talking about. So these are the phones that tend to
01:12:41have the best camera, you know, the best of the best. And those cameras have bigger sensors. That is a
01:12:50huge benefit when you're taking pictures in low light or really any situation. But a bigger sensor
01:12:57means it needs a bigger lens, hence a big camera bump. So there's a little bit of a caveat to this
01:13:05that outside of the US, this is maybe even a worse problem where some of the Xiaomi's of the world are,
01:13:14you know, betting on really big image sensors and big lenses and just sort of going all the way into,
01:13:22you know, the heck with it. A camera bump is going to be huge and people will just live with that.
01:13:27So I think what we have to deal with here is maybe a little bit better. And I do think the point about
01:13:35phones getting thinner, but the camera bump kind of remaining a problem is legitimate. I tested the
01:13:44Galaxy S25 Edge, which is a super thin phone. But that camera bump is maybe as thick as the phone
01:13:52itself. And something that really bothered me when I tested it is putting it on the table, as Osvaldo
01:14:00mentioned, does not sit flush on a table and it wobbles like all heck when you tap on it. This is made
01:14:08worse by my habit of not putting cases on phones, which I think is one of the solutions here. So I
01:14:17don't think the camera bumps are going away. Putting a case on your phone, depending on the phone in the
01:14:24case, you know, you can shop around a little bit for something that sort of evens out the camera bump.
01:14:29I think, you know, a pop socket or a case with a built in stand could help if you're really bothered
01:14:37by setting your phone down and having it wobble on the tabletop. Those would be things to look at if
01:14:43you haven't already. There's maybe good news coming or bad. I can't really tell. The rumors of the
01:14:53iPhone, the impending iPhone 17, show like an elongated camera bump that might help even out
01:15:03the wobble situation. I just tested the Pixel 10 Pro and Pixel phones have that long kind of elongated
01:15:15oval of a camera bump and it's big and chunky for sure. But when you set it on a table, it's a little
01:15:23bit of a stable stand for the phone. So it props it up a little bit, doesn't wobble around. So there
01:15:32are trade-offs there. Perhaps the iPhone is moving to something similar. It's still a big camera bump.
01:15:39So I think you've got to look at a case that's maybe going to help. If all else fails, I find a drink
01:15:46coaster is my friend. I am always putting my phone on a drink coaster when I'm sitting at the table and
01:15:54I'm tired of it wobbling around. So we've always got that Osvaldo. I'm sorry. I hope that helps. But
01:16:02maybe the help is in just saying it out loud and knowing that you're not suffering alone. So
01:16:09thanks for calling in. All right, that's it for the Vergecast today. To read more about the topics
01:16:15we covered, check the show notes or head to theverge.com. And if you like what we do here,
01:16:21consider buying a paid subscription. The show is produced by Eric Gomez, Brennan Kiefer,
01:16:26Travis Larchuk, and Andrew Marino. The Vergecast is a Verge production and part of the Vox Media
01:16:32Podcast Network. Jake Astronakis and the team will be here Friday to talk about the news. Thanks for
01:16:39listening.
Recommended
0:51
|
Up next
0:50
1:30:09
1:30:20
1:50:15
1:31:40
11:47
1:27:10
1:22:42
9:39
3:27
1:10
0:54
1:39
5:54
11:52
1:21:30
1:39:26
1:39:03
1:29:27
3:01
Be the first to comment