- 21 hours ago
Category
🗞
NewsTranscript
00:00It's been some time since I last saw you and in those days you were building Alexa from the ground
00:05up really. Let's start with Nova 2. This is the second generation of Amazon and AWS's frontier
00:13model and it's a year since the first iteration came out. This is a fancy question but in the
00:19sort of Amazon model taxonomy you put a lot of emphasis on reasoning. What is different with
00:26Nova 2 in that regard? Yeah, Nova 2 state-of-the-art frontier models. We introduced four of them
00:31today. A year back we announced Nova first generation of the models. Incredible adoption
00:37that we see with our customers. And then it's not been that we have only done Nova 1 but along
00:44the way we introduced many new models like Nova Sonic, Nova Premier. But today we announced
00:49our Nova 2 generation which has four models. Each are multimodal reasoning based models
00:57which means they process not just text as input but many other modalities. They're great for
01:02perception tasks, for everyday workflows, cost-effective reasoning style model where you can control
01:08the extent of the reasoning. You also have an upgrade to Nova Sonic and a first of its kind
01:13model called Nova Omni which not only generates text but also generates image output. The simple
01:20question a lot of people have is why did you need to have four variants and four different
01:25degrees of model performance in one release? Yeah, so we always find customers want different
01:32selection, different operating points. But let me get to the main thing here which please put
01:36it in context. If you think about it, the perennial challenge we have in the industry, a new model
01:42comes out, benchmarks look great, customers try it in production both internal and external
01:49customers that we have and we find the consistent feedback, oh it doesn't work in our environment
01:55as we would think or in our production setting. So that's where the reality sinks in. There is
01:59a simple reason for that because these frontier models as great they are, they do not know your
02:04domain, your proprietary workflows which sit behind your walls. So we ask the question, how can
02:10we build something that knows your systems, your data? And for that the real answer was
02:17that everyone is seeking a frontier model that's expert in your domain. Now you have a few choices
02:24right now, all of them are suboptimal, you can start fine-tuning on the edges of frontier model,
02:29you can take an open weights model. Suboptimal why? I'll get into that, so you can fine-tune only on the edges
02:35for instance because you have these adapter-based model training which doesn't really give you the
02:40full expertise because these models are already fully built. So you can't steer them to your domain
02:45and when you try to steer it to your domain it forgets a lot of its general properties that were
02:51remarkable. So then you're left with this ultimate choice which is do I build my own from scratch but
02:57that's prohibitively expensive so that's why we brought NovaForge because now you have a way of
03:03building your own frontier model starting with every stage of the model development from pre-training
03:09to mid-training and post-training with one unique thing which is you can now infuse your data and
03:17pre-mix with Amazon curated data sets. So let me jump in here, I'm not going to be to pretend to be smarter
03:22than I am. You can have a foundation model with tens of billions, hundreds of billions and parameters
03:28but the issue that you're outlining is that if it has no relevance to you in its data set it has a
03:34limited utility. So let's go to NovaForge. This is the ability to build a model using data sets from
03:43within your own company, simple layman's intention. Just go a bit further on what you're trying to solve
03:48for here and actually if it's as good as you say the commercial importance of it. Yeah because the
03:54general intelligence model are not it's not like intelligence is a monolith process right so the
04:01intelligence to be really useful in your setting you need to plug in your data your knowledge your
04:06system workflows and that starts with the earliest stages of the training that's how we grow a lot of
04:12our foundational capabilities we are the most efficient if we have learned the domain as early as possible
04:18that's what we're trying to do with NovaForge you have every multiple checkpoints available for you
04:23to make Nova your own which we actually call a novella right and you can name it whatever you want
04:29but this is becomes your model which knows your environment that's optimized as a variant that you
04:34can build your applications and agents for and the beauty is that it preserves the foundational properties
04:40because the core data that gives that frontier intelligence is also mixed with your proprietary domain
04:46knowledge. I always am a bit hesitant to give these kind of behind the scenes moments but before we
04:51we started this live conversation we were debating is Amazon or AWS a full stack AI company
04:58the debate around it is AWS has always been number one cloud computing in terms of scale for both compute
05:06capacity storage and what's available right is it the number one AI company is now the question
05:13and what unites those two debates is cost and efficiency yes what is different with the Nova
05:202 generation and with NovaForge from that perspective absolutely first AWS I'm glad people think of it as
05:27infrastructure because that's where everything starts but to us it's always been a full stack
05:31company where you have the infrastructure layer the model and AI as a service at the middle layer and at
05:36the top layer you have multiple applications that are serving our customers it's always been a full stack
05:41uh with Nova as you said uh we are making really democratizing the AI for everyone where you everyone
05:50can have their frontier model that knows their domain and with that the beauty is that you can uh the cost goes
05:57down for building your applications on top of this model that's your variant because you can start with a
06:03model that can actually be distilled from a large teacher model or a smaller model that can be still very
06:09performant compared to other large giant models so that also gives you a huge cost level and this is
06:15why forge is great because it's making AI more affordable as well for everyone what did the benchmarks uh
06:21by which you hold yourselves to account on that um you know you i think you've discussed with my colleague
06:27matt day about Nova one generation where it ranked in the benchmarks the real test is in real world deployment
06:34but is it as simple as saying cost per token latency multimodal latency how do you judge your efforts
06:40yeah first of all public benchmarks are a good indicator but they're not sufficient the real indicator is
06:46performance in your real world environments and that comes down to accuracy cost latency and that's
06:52why we refer to this as uh price performance Nova models are industry leading in price performance
06:58because we have optimized for all three together and that's incredibly hard and with forge now we are
07:03giving you those knobs to optimize for your price performance deeds because uh your applications may
07:10have a different scaling requirement different kind of use cases and now you have the same knobs that
07:15our internal teams have and that's why i love forge right internally is agi a goal and a remit that
07:24you've been given by this company to achieve yeah first of all i wish agi was thought about as a practical
07:30thing not a sensationalizing element to me we are really focused on building this generally intelligent
07:39model that can be forged into these different specialists that do useful things in your environment
07:45and starting with the digital environment is great things will happen in the physical environment as
07:49well this is why we have giant robotics efforts as well internally and externally you see a lot of people
07:54working in that area so i'm a very optimist about practical agi that will help everyone in humanity
08:02very quick fact check nova forge i've seen a number of a hundred thousand dollars per year what does that
08:07represent it's a per desk it's a per training exercise explain that the business model of it yeah it's for
08:14by your company's account i mean we can uh the key is that i would focus on not that amount because you get
08:19to think about what you're getting for it you're getting the deepest access possible in what i think
08:24of as open training paradigm where you're also getting access to frontier scale data mixing that
08:31makes these properties of the model be not just general purpose but also more specialized in your domain
08:37so the value is incredible and uh and then uh i think you will see that this kind of a value
08:44uh that amount is actually minuscule in the context of things when we first met you were
08:50building alexa early generations of of echo dots um i as you know have a have an echo device in most rooms
08:59in my house and i have been able to use alexa plus but my interactions with alexa the latest generation are
09:06not what my interactions are with various other tools from open ai anthropic google particularly in voice
09:14you know i think about how i use chat gpt voice but there is improvement can you because it's still
09:19within the portfolio for you what comes next with alexa and how are you going to take the work you've
09:24done with nova to make improvements because alexa is built on nova among other models so alexa plus
09:31i'm super excited about if if you haven't used it i urge you to use it a lot now it's a remarkable
09:36upgrade in over uh the alexa of the uh the one that you've been using for a while it's completely
09:43react detected from the ground up uh but the customer reception has been great we are seeing
09:48personally in my usage it's a much more useful and very natural ai and what all it can do
09:55is still i haven't seen something that can do everything alexa plus can do
Be the first to comment