- 6 months ago
Join us for a clear, engaging look at x86-64 program segments! We break down the text, data, and BSS sections, explore stack vs. heap memory, and show a real stack overflow in C++. Learn why segmentation faults happen, how virtual memory works, and tips to avoid crashes. Whether you're new to assembly or leveling up your coding skills, this video is packed with insights to boost your understanding of low-level programming. Subscribe for more coding deep dives!
Introduction to x86-64 program segments 00:00:01
Understanding segmentation faults 00:00:12
Importance of segments in assembly 00:00:32
Overview of assembly program structure 00:01:09
Defining segments in Yasm assembly 00:01:49
Data section: Initialized global variables 00:02:20
BSS section: Uninitialized variables 00:03:23
Text section: Code and functions 00:03:59
Global and extern function declarations 00:04:41
Stack: Local variables and return addresses 00:07:11
Stack overflow example in C++ 00:08:48
Heap: Dynamic memory allocation 00:11:31
Stack vs heap memory growth 00:11:56
Virtual memory and memory allocation 00:14:00
Demonstrating memory overflow with heap allocation 00:16:59
Summary of segments and their purposes 00:17:59
Closing remarks and call to subscribe 00:18:47
Thanks for watching!
Find us on other social media here:
- https://www.NeuralLantern.com/social
Please help support us!
- Subscribing + Sharing on Social Media
- Leaving a comment or suggestion
- Subscribing to our Blog
- Watching the main "pinned" video of this channel for offers and extras
Introduction to x86-64 program segments 00:00:01
Understanding segmentation faults 00:00:12
Importance of segments in assembly 00:00:32
Overview of assembly program structure 00:01:09
Defining segments in Yasm assembly 00:01:49
Data section: Initialized global variables 00:02:20
BSS section: Uninitialized variables 00:03:23
Text section: Code and functions 00:03:59
Global and extern function declarations 00:04:41
Stack: Local variables and return addresses 00:07:11
Stack overflow example in C++ 00:08:48
Heap: Dynamic memory allocation 00:11:31
Stack vs heap memory growth 00:11:56
Virtual memory and memory allocation 00:14:00
Demonstrating memory overflow with heap allocation 00:16:59
Summary of segments and their purposes 00:17:59
Closing remarks and call to subscribe 00:18:47
Thanks for watching!
Find us on other social media here:
- https://www.NeuralLantern.com/social
Please help support us!
- Subscribing + Sharing on Social Media
- Leaving a comment or suggestion
- Subscribing to our Blog
- Watching the main "pinned" video of this channel for offers and extras
Category
🤖
TechTranscript
00:00Hi there.
00:01Let's talk about x86-64 programs and their segments and the purpose of the segments.
00:12Probably when you've been programming in the past you've seen segmentation fault errors.
00:17If you've accidentally crossed the bounds of some appropriate memory location, if you
00:22have an array and you try to index it way outside of bounds or something, you might
00:26have seen a segfault.
00:28Well it's basically come from the idea that you have crossed a segment boundary.
00:32Segments used to be really, really important, they're a little bit less important now that
00:35we have virtual memory, I'll just talk a little bit about them.
00:38I'm going to be discussing this from the purpose of an assembly program, just because even
00:44in higher level languages, your languages will typically be compiled down to assembly first
00:49before they actually hit machine code and your actual executable binary.
00:54So if you're not interested in YASM assembly, I think this could still be worth it for you
01:00to watch because you'll be able to see the different segments and just kind of get a better
01:04understanding of what they are.
01:06Okay.
01:07So I'm going to actually write a very, very, very quick program in assembly.
01:11It's not going to really do anything.
01:12I just want you to show, I just want you to see the segments that are involved.
01:17So for starters, let's just pretend that I'm writing an assembly program here.
01:22We'll say that semicolons are comments.
01:24This is a comment just so you know what I'm about to type.
01:28Perhaps actually this is not the best editor.
01:29I don't know why I pulled this up.
01:31Let me get Genie here.
01:32It's a little bit better.
01:33Genie.
01:34There we go.
01:37Maybe I'll save it somewhere.
01:39I'll say segments assembly and then now the comment gets highlighted.
01:45Okay.
01:46So in assembly, in YASM assembly, we denote various segments by using the section keywords.
01:52So we'll say section and then dot something indicating the type of segment that we're defining.
02:00So section text and then up here, we'll have another one section dot BSS.
02:07And you know, if you already program assembly, you know that somewhere in here is where the
02:10code goes right between these sections.
02:14So just a quick recap of what the data section is.
02:17It just is where you store a bunch of global variables that you can name and initialize
02:21at the same time at the very top.
02:24Globals are not great, but sometimes you want to do that in assembly depending it's, it's better
02:28if you stick all your variables on the stack.
02:30Like if they're a local variable, it's a little bit better.
02:33Think of higher level languages.
02:35You typically want to avoid global variables if you can possibly avoid them and make everything
02:39else a local variable or a member variable.
02:42So that means somewhere else other than data, probably the stack, but, or the heap.
02:49Okay.
02:50You know, this is like the basic pattern.
02:51We'll say, you know, my variable.
02:54And then we'll decide to say what data size it is.
02:57In this case, it's DB for, for one byte of data.
03:00You could put a, I think DD in there or a DW for a word.
03:04Um, sorry, sorry, DW would be for a word.
03:07And then I think DD would be a double and then DQ would definitely be a quad word.
03:12So here quad word is eight bytes and then I can sort of initialize it to some number.
03:17And now I have a global variable.
03:18So not like a huge deal.
03:21The BSS section is basically where you have uninitialized, uh, variables, but there you
03:29can make huge amounts of variables.
03:31You can make an array of variables.
03:33So if I say my array, something like that, I could put resource queue to say, I want the
03:38type of one item to be a quad word.
03:41So, you know, eight bytes per item.
03:43And then I could just say, I want, you know, a hundred thousand items or something.
03:49And I'll double check that after I stopped recording this video and correct it if I'm
03:54wrong.
03:55It's been a while since I actually typed BSS here.
03:58And then, um, in the text section, uh, this is where you put code and functions and things.
04:02You typically want to have at least one entry point somewhere.
04:04Uh, you know, if you want to make like, if you want to make this the entry point of your
04:08entire program, you'll probably do global main and then you'll do main.
04:13And then this assumes that you use the GCC libraries and just sort of return from the
04:18label sort of makes it a function assuming you didn't ruin any of the other data.
04:24So those are the three sections that we can start off looking at the data section, the BSS
04:30section and the tech section.
04:32It's important to note that besides marking functions as global so that they're accessible
04:37uh, to other segments or other modules.
04:41We also mark functions, uh, and labels that we would like to access that are in other modules.
04:45So if we're using the GCC libraries, we could say something like global print F and then right
04:50away we're able to call the print F function.
04:53Or say if you have another function in another module, like a different source code file that's
04:57compiled somewhere else, we could say, you know, my other function.
05:05Like that, just mark it as global or sorry, not global.
05:08Extern.
05:10Global is when you have the function in the current module and you want it to be available
05:15to other modules.
05:16Extern is when the function or label is in another module and it's already been made available
05:21to you, but you want your current module to know that it can access it for the purposes
05:26of assembling the program.
05:29So that means inside of here we could do something like call.
05:32We could say, let's call my other function.
05:39That will work if the other function is actually a function with a return statement and it respects
05:43the ABI, you know, it pushes registers that it's going to modify if they're callly saved.
05:48If it's not another function, maybe we'll say extern my other jump point.
05:55So like not a proper function, but just something that you might want to jump to, then instead
06:01of using call, we would use a jump, jump, you know, and so forth.
06:07So those are the basic sections.
06:09Um, let's see, for the BSS, I definitely talked about token size and count.
06:15And then there's resource B, resource W, resource D, resource Q for the different sizes.
06:20So basically just take all the different sizes you could put in the regular data section and
06:25assume that that could be popped on to the end of RES inside of the BSS section.
06:30If you're interested, this is a segments video, not an assembly video.
06:36Another segment, another segment that we're not showing here in this source code is the
06:39actual stack.
06:40So probably, I guess the best way that I could show it is, I don't know, by messing around
06:48with the stack pointer, increase, uh, oh, RSP, why did I think it was the instruction pointer?
06:58We could increase the stack pointer and then decrease the stack pointer, but it doesn't
07:02really help a whole lot.
07:04But, uh, in another video, I'm going to talk more in depth about what the stack is and how
07:08to use it and such, but just keep in mind that the stack is where local variables are stored
07:13and, um, you can use it in assembly and C++ and other higher level languages.
07:18You can even make virtual stacks in any type of program almost, uh, to just sort of, you
07:23know, have a stack like structure.
07:26But in C++, if you make any local variable, then it's usually going on the stack.
07:31And the stack tends to have a fixed, um, size.
07:35And so if you sort of put too much data on the stack, you, you risk a stack overflow, which
07:41will crash your program, uh, besides local variables, the stack will hold, um, other information
07:46that the current scope needs like its return address.
07:50So when we, when we make a function and then let's say we, we call another function inside
07:54of it, uh, actually just up here, even when we call another function on line 22, the stack
08:00will receive the return address of where we currently are so that later when we return
08:05from the other function, you know, the program knows where to jump back and forth.
08:10It's actually under the hood, just jump instructions.
08:12When you do call, it's, it's not, it's not like a special thing.
08:16It's just jumping to another location and then pushing the return address on the call
08:22stack.
08:23So that's other stuff that the stack holds, um, which means if you have, uh, like an endless
08:28or like a infinite recursion happening in your function where just a function calls itself
08:34over and over and over again, or you have some complicated call graph and you end up calling
08:38too many functions without ever returning, uh, you could overflow the stack.
08:43So that's not good.
08:45Let me show you a quick example that I, that I'm going to make for you real fast.
08:48Oh shoot.
08:49Did I actually make this?
08:50Where is the, uh, it's on the desktop.
08:55Okay.
08:56I guess I'm going to make more on the desktop.
08:58Let's do a stack overflow.
09:01Let me do desktop here.
09:03Let's do stack overflow that CBP.
09:07I'm just going to make a quick program, uh, in C plus plus that doesn't really do anything
09:11except overflow the stack.
09:13So I just want you to know the stack can overflow.
09:16I'm going to make a function called F and F just calls itself.
09:21So that's going to be an infinite recursion.
09:23And that means, you know, every single time F calls itself, the return address of like
09:29line five is going to get pushed onto the stack again.
09:32So it's just going to get pushed and pushed and pushed and pushed the stack will overflow
09:35faster.
09:36If I had local variables, because every scope that gets called, even if it's the same function
09:41name, it'll have its own allocation on the stack.
09:44Um, so if I just sort of like call this from main and then I'll just need to make a little,
09:51uh, a little make file that just kind of compiles.
09:55I don't really even need to make a make file.
09:58I could do a build script, but I'm going to do it anyway.
10:00We'll say G plus plus, uh, standard equals C plus plus 23 show all warnings show pedantic.
10:09Uh, the input file will be stack overflow CPP.
10:14And then the output file will be main.
10:16And then assuming that succeeds, I'll just run main.
10:19So this is like not a make file video.
10:22I have other videos for make files.
10:25Uh, let's see, let me get to the desktop.
10:29If I run the make file with just the word make, you should see a stack overflow.
10:34We should also see a warning about the fact that there's infinite recursion.
10:39No, we didn't see a warning.
10:44That's funny.
10:45I must've done something wrong or different because on a different computer,
10:48not too long ago, I did the same thing and I saw a warning.
10:51Okay.
10:52Well, thanks for not warning me.
10:53But anyway, you can see that the, uh,
10:55we have a segmentation fault here because the stack overflow,
10:58we just made a stack that was too huge.
11:00This is a reminder that the stack is not necessarily as dynamic as the heap.
11:05Um, you, you can definitely overflow it.
11:08It's not really meant to store gigantic amounts of data.
11:11What am I talking about when I say the heap though?
11:14So just keep in mind that.
11:16Whenever you allocate dynamic data, well, maybe,
11:21well, what do you think?
11:23Maybe, maybe inside of, instead of calling F let's call G.
11:28The heap is where you allocate a dynamic data.
11:31So anytime you use the malloc operator in C or the new allocator in C plus plus,
11:37you're, you know, creating like a dynamic allocation of data.
11:40I'll do a new integer and then I'll do like a thousand integers.
11:48So this is basically going to allocate a thousand new integers in the heap,
11:52which is a segment that is right up against the stack.
11:55I should mention also that the stack is important.
11:59The stack, uh, its memory locations go down as the stack grows.
12:03Keep that in mind because typically when you, when you imagine a stack,
12:08I'll, I'll make more videos in the future so you can visualize the stack better.
12:12But most of you who know what a stack is already,
12:14uh, you imagine that the stack grows upward visually, right?
12:17But in your computer, the memory locations of the stack grow downward.
12:22So if I add something to the stack,
12:24then the new stack head pointer is actually going to be a lesser address.
12:29And the reason for that is the stack in the heap,
12:31they grow in the same direction towards the same unallocated space.
12:36We'll call that space, uh, the unallocated memory space or the unmapped memory space.
12:43And it's just, you know, a segment stack in the heap.
12:45And then there's the unallocated space that they grow towards each other.
12:48If they ever meet, uh, then, then your program is out of memory and it'll crash.
12:53But the heap actually can be dynamically allocated to store tons of, of RAM.
12:57I'm probably about to crash this computer.
13:02So maybe this video won't finish recording.
13:04So maybe I'll try to crash this at the very end.
13:06I just wanted you to see, let, let me, let me do this.
13:09I'll say while true.
13:10Um, we'll just allocate a new integer.
13:15This is kind of bad form because when you allocate, you know, a new variable somewhere,
13:19you want to have access to it after the allocation.
13:22You would usually send that into a pointer.
13:24You'd say like, you know, uh, integer pointer P equals new in something like that.
13:28Right. But, uh, in this case, I don't really care.
13:30I'm not going to, I'm not going to deallocate that memory.
13:32I just want you to see how fast the memory can grow.
13:35When we did, uh, the stack overflow seemed like the program died pretty quickly,
13:40but, uh, here we're going to be able to see the memory boom pretty, pretty big.
13:45Um, before the whole computer just starts to, you know, fall to its knees.
13:49So I'm going to do a call to G here.
13:51And then I won't actually run this until I'm ready to crash this whole video.
13:55Let's see. What else can I say?
13:57Uh, oh, the reason that the heap can, can grow so much larger than the stack is our programs.
14:06Now in, in, in the modern era, they use virtual memory.
14:08So it's not like when we allocate, uh, let's say we, uh, we're, we're accessing a memory location via a pointer inside of our program.
14:17It's not that the pointer has the absolute memory location through the whole system.
14:22Like it doesn't, it's not able to name locations outside of the current program.
14:27Every program sees its own start memory location as just zero.
14:31And then under the hood, there are some extra tricks so that when you try to access a memory location,
14:36let's say memory location 100, because you've made, I don't know this many integers or whatever.
14:41Um, then under the hood, the, the computer will know what is the start offset of the program's memory.
14:50So maybe the start offset is 1000.
14:52It's like totally unrealistic, but just pretend the start offset is 1000.
14:56Then that means if it's trying to access a pointer with memory location 100,
14:59it just gets added to the offset, the virtual offset.
15:02So it'll be a thousand and 100 is the real, you know, physical memory location,
15:06or at least at that point, it'll probably go to the paging system, but that's, that's another video.
15:11Um, so just know that it's virtual, which means the program thinks it has the full range of,
15:18of the 64 bit memory address space.
15:21Even if you don't even have that much memory on your computer, which there's no way,
15:24there's no way you have so much memory that it's going to overflow or even meet 64 bits at all.
15:29But the program thinks it can.
15:31So it's just going to grow and grow and grow until you actually run out of physical memory.
15:35And then eventually, uh, probably your program is going to start going to the page file or the,
15:40or to the swap file.
15:41And then when you run out of that, then your whole computer probably is going to start crashing.
15:46Um, to be honest, it'll probably start crashing as soon as it hits the swap file,
15:50because at that point, none of the other programs really have enough leftover memory to,
15:55to breathe, to do extra allocations and things.
15:58So everything's going to start slowing down.
16:00It's going to be pretty bad.
16:01Um, well, I think that's all I really needed to say.
16:05This is not supposed to be a really long video.
16:07Let me, uh, let me show you this crasher.
16:10I'm going to say goodbye now.
16:11Thank you so much for watching the video.
16:13I hoped you learned a little bit and had some fun too.
16:15And, and that way I've already said goodbye in case the program crashes.
16:19We'll see if it actually crashes the computer.
16:21So I'm going to open up this and then I'm going to do maybe H top so that I can see memory.
16:28So if you kind of like look at H top real fast, this is just a Linux command line tool that you can install.
16:35It's pretty cool.
16:36The heck's going on here.
16:37Oh, that's my recorder.
16:38Uh, you can see here's the memory allocation.
16:41So I'm using two gigs of eight gigs on this virtual machine.
16:45And then, uh, I'm not using any swap space, which means I have not exhausted all my memory.
16:51And, and I have not been forced to go to the disc for like additional memory, but in a second, we'll see that will probably change.
17:00So I'm going to do make just to run that.
17:03It looks like it's stuck, but if you keep looking at the memory allocation, Oh, it already hit eight gigs.
17:08Now it's into the swap already.
17:10Probably.
17:11Okay.
17:12I forgot about that.
17:14The out of a memory killer, the OOM successfully killed the program.
17:20Thank goodness.
17:21Or the whole computer would have been down.
17:23I always forget that that's going to happen lately, but, um, you can see it spiked, right?
17:28It went right up to the full physical memory allocation of 7.75 gigs.
17:32And then the swap file went all the way up to two gigs.
17:35You can see that the swap file still has about 700 megs inside of it.
17:40That just means that there's memory that isn't really being accessed.
17:43So the operating system doesn't feel like it's necessary to pull it back out of swap and put it into RAM.
17:48But, um, I wonder if the video skipped when I did that or stuttered or anything.
17:53I'm interested to find out now.
17:55Okay.
17:56So that's the end of this video.
17:57I hope you have a decent basic understanding of segments.
18:01Uh, and then, you know, like the segments and their purpose, you know, the heap and the stack, they grow together.
18:06Uh, the heap can dynamically, uh, allocate to grow bigger.
18:09Notice how it went up to eight gigs just now, basically, whereas the stack will crash a lot faster.
18:14If the stack and the heat heap meet, then you've ran out of memory at that point.
18:18You know, you probably want to like resize the heap or the stack just crashes.
18:22And, um, then we've got the tech section of the program and the data section and the BSS section of the program.
18:29Uh, these sections are a little bit less relevant in the modern era because we have, you know, virtual memory and stuff.
18:37But, uh, when you see a segmentation fault, that definitely means you have tried to access outside of your segment and it's a no go.
18:44Okay. Thank you for watching this video.
18:47Uh, I will see you at a later date in time.
18:50Hey everybody.
18:53Thanks for watching this video again from the bottom of my heart.
18:55I really appreciate it.
18:56I do hope you did learn something and have some fun.
18:59Uh, if you could do me a please, a small little favor, could you please subscribe and follow this channel or these videos or whatever it is you do on the current social media website that you're looking at right now?
19:10Um, it would really mean the world to me and it'll help make more videos and grow this community.
19:16So we'll be able to do more videos, longer videos, better videos, or just, I'll be able to keep making videos in general.
19:22So please do, do me a kindness and, uh, and subscribe.
19:26You know, sometimes I'm sleeping in the middle of the night and I just wake up because I know somebody subscribed or followed.
19:31It just wakes me up and I get filled with joy.
19:33That's exactly what happens every single time.
19:35So you could do it as a nice favor to me or you could, you could troll me if you want to just wake me up in the middle of the night, just subscribe.
19:41And then I'll, I'll just wake up.
19:43Uh, I promise that's what will happen.
19:44Also, uh, if you look at the middle of the screen right now, you should see a QR code, which you can scan in order to go to the website, which I think is also named somewhere at the bottom of this video.
19:54And it'll take you to my main website where you can just kind of like, see all the videos I published and the services and tutorials and things that I offer and all that good stuff.
20:03And, uh, if you have a suggestion for, uh, uh, clarifications or errata or just future videos that you want to see, please leave a comment.
20:14Or if you just want to say, Hey, what's up?
20:16What's going on?
20:17You know, just send me a comment, whatever.
20:19I also wake up for those in the middle of the night.
20:21I get, I wake up in a cold sweat and I'm like, it would really, it would really mean the world to me.
20:26I would really appreciate it.
20:27So again, thank you so much for watching this video and, um, enjoy the cool music as, as I fade into the darkness, which is coming for us all.
20:51I'm hot.
20:52Cause I adjusts my favorite parts of my life.
20:55I just feel, this all I make.
21:05I'm sorry.
Comments