Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
SlideShare a Scribd company logo

Unix memory management

•Download as PPT, PDF•
•
T
Tech_MX

This document provides an overview of UNIX memory management. It discusses the history of UNIX and how it evolved from earlier systems like Multics. It describes swapping as an early technique for virtual memory management in UNIX and how demand paging was later introduced. Key concepts discussed include page tables, page replacement algorithms like two-handed clock, and the kernel memory allocator.

Related slideshows

Introduction to System Calls
Introduction to System CallsIntroduction to System Calls
Introduction to System Calls
 
Presentation on Segmentation
Presentation on SegmentationPresentation on Segmentation
Presentation on Segmentation
 
Virtual memory presentation
Virtual memory presentationVirtual memory presentation
Virtual memory presentation
 
1 of 35
UNIX MEMORY MANAGEMENT 1
AGENDA  Introduction  History of UNIX  Swapping  Demand Paging Page Replacement Algorithm  Kernel Memory Allocator  Conclusion 2
Introduction  UNIX is a portable, multi-tasking and multi-user operating system. – Portable: runs on many different hardware architectures (Intel x86 and IA-64, Alpha, MIPS, HP PA-RISC, PowerPC, IBM S/390, SPARC, Motorola 680x0, etc.). – Preemptive multi-tasking: several programs can run at the same time (time slices, interrupts, and task switching). – Multi-user: many users can share the computer system at the same time. 3
Other Features… • Uses a simple, uniform file model which includes devices and access to other services in a flexible, hierarchical file system. • Written in a high-level language (“C”) making it easy to read, understand, change and port. • The command prompt is a simple user process, the Unix shell, which is also a convenient job programming language. • Includes support for regular expressions which are convenient for complex searching. 4
History of UNIX • 1964 joint project between AT&T Bell Labs, GE, and MIT to develop a new OS. • Goal : develop an OS that could provide computational power, data storage and the ability to share data among multiple users. • Result: Multiplexed Information & Computer Service - MULTICS. 5
• 1969 Bell Labs withdraws from group. • Two Bell Lab scientists, Ken Thompson and Dennis Ritchie, continue research. They were still left without a “Convenient interactive computing service”*. • At the same time Ken Thompson wrote a game “space travel” in Fortran to run on GECOS OS. • The spaceship was hard to control and it was expensive to run. He was told to get the game off his work computer. 6
• Thompson ported the game to a little used PDP-7 computer. • Unics (later Unix) was born as a pun on Multics. • Dennis Ritchie developed “B” . Then wrote “C” a compiled language. • In 1973 entire OS ported to “C”. • 1991 Linux 0.02 is first released to the public. • 1994 Linux 1.0 is released. 7
Memory Management  UNIX is machine independent so its memory management scheme will vary from one system to next.  Early versions of UNIX used variable partitioning with no virtual memory scheme.  Current implementations of UNIX make use of paged virtual memory.  There are two memory management schemes :  Paging System  Kernal Memory Allocator 8
Virtual Memory 9
Memory Management policies • Swapping – Easy to implement – Less system overhead • Demand Paging – Greater Flexibility 10
Swapping • The process of moving some pages out of main memory and moving others in, is called swapping. • A page fault occurs when the CPU tries to access a page that is not in main memory, thus forcing the CPU to wait for the page to be swapped in. • Since moving data to and from disks takes a significant amount of time, the goal of the memory manager is to minimize the number of page faults. 11
• Swap Space - Disk memory used to hold data that is not in Real or File System memory. Swap space is most efficient when it is on a separate disk or partition, but sometimes it is just a large file in the File System. • Allocation of both main memory and swap space is done first-fit. • A page fault occurs when the CPU tries to access a page that is not in main memory, thus forcing the CPU to wait for the page to be swapped in. 12
• Since moving data to and from disks takes a significant amount of time, the goal of the memory manager is to minimize the number of page faults. • When the size of a process' memory image increases (due to either stack expansion or data expansion), a new piece of memory big enough for the whole image is allocated. • If no single piece of main memory is large enough, the process is swapped out such that it will be swapped back in with the new size. 13
• Decisions regarding which processes to swap in or swap out are made by the scheduler process (also known as the swapper). • A process is more likely to be swapped out if it is idle or has been in main memory for a long time, or is large ; if no obvious candidates are found, other processes are picked by age. • A process is more likely to be swapped in if its has been swapped out a long time, or is small. 14
15
Allocating Swap Space Address Unit Allocate 100 unit 1 10000 101 9900 Map Allocate 50 unit Allocate 100 unit 251 9850 151 9750 16
Freeing Swap Space Address Unit 50 unit free at 101 251 9750 101 50 Map 251 9750 Case 1: Free resources fill a hole, but not contiguous to any resources in the map 17
Freeing Swap Space Address Unit 50 unit free at 101 101 50 251 9750 251 9750 Map 100 unit free at 1 1 150 Case 2: Free resources fill a hole, 251 9750 and immediately precedes an entry in the map 18
Freeing Swap Space Address Unit 101 50 50 unit free at 101 251 9750 251 9750 Map 100 unit free at 1 1 150 Allocate 200 unit 1 150 451 9550 251 9750 300 unit free at 151 Case 3: Free resources fill a hole, and completely fills the 1 10000 gap between entries in the map 19
Demand Paging • Demand paging to unix with BSD(Berkley system) which transferred memory pages instead of process to and from a secondary device. • When a process needs a page and the page is not there, a page fault to the kernel occurs a frame of main memory is allocated, and the process is loaded into the frame by the kernel. 20
Page Table Page Frame Age Copy on Modify Reference Valid Protect Number write • frame # contains the physical frame where the virtual page is stored • age is processor dependant, and is meant to maintain how long it has been since the page was accessed. • Copy on Write store the copy on write bit, which is used in UNIX systems to, among other things, render fork efficient. 21
• Dirty is a single bit that indicates whether a page has been modified since last swapped in (the opposite of dirty is clean, and a clean page need not be written out to disk if swapped). • Ref contains the usage information necessary for a CLOCK- style algorithm. • Valid is the standard UNIX jargon for resident. A valid page is in main memory, an invalid one is swapped out. • Protect contains the permission information for the page and is also hardware dependant. 22
Disk block descriptor Swap device Number Device Block No. Type of storage • The disk block descriptor contains the information mapping a virtual page to a spot on disk. • The OS maintains a table of descriptors for each process. 23
• Device # is basically a pointer to the disk that this page was swapped to. • Block # is the actual block that the page is stored on. This is why most UNIX systems prefer to have a separate swap partition, so that the block size can be set to the page size. • Type specifies whether the page is new or pre-existing. This lets the OS know if it has to clear the frame first. 24
Page frame data table Page State Reference Logical Block Pf data Count device number pointer • The page frame data table holds information about each physical frame of memory (indexed by frame number). • This table is of primary importance for the replacement algorithm. 25
• Page state indicates whether or not the frame is available or has an associated page (i.e. whether its been allocated to a process or not). • Ref. Count holds the number of processes that refer to this page (remember, processes can share the same physical page). • Logical device contains the device number of the disk that holds a physical copy of the page. 26
• Block # holds the block number on that disk where the page data is located. • Pfdata pointer is a pointer field that is used to thread a singly-linked list of free frames through the frame table. If the page is free, this points to the next free page (useful for free list-style allocation). 27
Swap use table Reference Page/storage count unit number • Reference Count : Number of page table entries that point to a page on the swap device. • Page/storage unit number : Page identifier on storage unit 28
Page Replacement Algorithm • The Page frame data table is used for page replacement. • All of the available frames are linked together in a list of free frames available for bringing in pages. • The two-handed clock algorithm uses the reference bit in the page table entry for each page in memory that is eligible (not locked) to be swapped out. • This bit is set to 0 when when the page is first brought in and set to 1 when the page is refernced for a read or write . 29
Two-Handed clock Page Replacement Algorithm 30
• Two parameters determine the operation of the algorithm: – Scanrate: The rate at which the two hands scan through the page list, in pages per second. – Handspread: The gap between fronthand and backhand • These two parameters have default values set at boot time based on the amount of physical memory. 31
• The parameter varies linearly between the values slowscan and fastscan as the amount of free memory varies between the values lotsfree and minfree. • The handspread parameter determines the gap between the fronthand and the backhand and therefore, together with scanrate,determines the window of oppurtunity to use a page before it is swapped out due to lack of use. 32
Kernal Memory Allocator • Kernel memory allocator: provides buffers of memory to various kernel subsytems. Evolution Criteria : – must be space-efficient i.e. minimize wastage. – Can be measured by utilization factor – must be fast – must have a simple programming interface – should not force to free the entire allocated area all at once – must guard against the wastage of memory – must be able to interact with the paging system 33
Advantages • easy to implement • not restricted to memory allocation • no wastage of space • can release any part of the region • allows reuse of memory by coalescing 34
References Books : ********* 1. Os Internals & design --- William Stallings 2. The design of the unix operating system --- Maurice J. Bach prentice hall 35

More Related Content

Unix memory management

  • 2. AGENDA  Introduction  History of UNIX  Swapping  Demand Paging Page Replacement Algorithm  Kernel Memory Allocator  Conclusion 2
  • 3. Introduction  UNIX is a portable, multi-tasking and multi-user operating system. – Portable: runs on many different hardware architectures (Intel x86 and IA-64, Alpha, MIPS, HP PA-RISC, PowerPC, IBM S/390, SPARC, Motorola 680x0, etc.). – Preemptive multi-tasking: several programs can run at the same time (time slices, interrupts, and task switching). – Multi-user: many users can share the computer system at the same time. 3
  • 4. Other Features… • Uses a simple, uniform file model which includes devices and access to other services in a flexible, hierarchical file system. • Written in a high-level language (“C”) making it easy to read, understand, change and port. • The command prompt is a simple user process, the Unix shell, which is also a convenient job programming language. • Includes support for regular expressions which are convenient for complex searching. 4
  • 5. History of UNIX • 1964 joint project between AT&T Bell Labs, GE, and MIT to develop a new OS. • Goal : develop an OS that could provide computational power, data storage and the ability to share data among multiple users. • Result: Multiplexed Information & Computer Service - MULTICS. 5
  • 6. • 1969 Bell Labs withdraws from group. • Two Bell Lab scientists, Ken Thompson and Dennis Ritchie, continue research. They were still left without a “Convenient interactive computing service”*. • At the same time Ken Thompson wrote a game “space travel” in Fortran to run on GECOS OS. • The spaceship was hard to control and it was expensive to run. He was told to get the game off his work computer. 6
  • 7. • Thompson ported the game to a little used PDP-7 computer. • Unics (later Unix) was born as a pun on Multics. • Dennis Ritchie developed “B” . Then wrote “C” a compiled language. • In 1973 entire OS ported to “C”. • 1991 Linux 0.02 is first released to the public. • 1994 Linux 1.0 is released. 7
  • 8. Memory Management  UNIX is machine independent so its memory management scheme will vary from one system to next.  Early versions of UNIX used variable partitioning with no virtual memory scheme.  Current implementations of UNIX make use of paged virtual memory.  There are two memory management schemes :  Paging System  Kernal Memory Allocator 8
  • 10. Memory Management policies • Swapping – Easy to implement – Less system overhead • Demand Paging – Greater Flexibility 10
  • 11. Swapping • The process of moving some pages out of main memory and moving others in, is called swapping. • A page fault occurs when the CPU tries to access a page that is not in main memory, thus forcing the CPU to wait for the page to be swapped in. • Since moving data to and from disks takes a significant amount of time, the goal of the memory manager is to minimize the number of page faults. 11
  • 12. • Swap Space - Disk memory used to hold data that is not in Real or File System memory. Swap space is most efficient when it is on a separate disk or partition, but sometimes it is just a large file in the File System. • Allocation of both main memory and swap space is done first-fit. • A page fault occurs when the CPU tries to access a page that is not in main memory, thus forcing the CPU to wait for the page to be swapped in. 12
  • 13. • Since moving data to and from disks takes a significant amount of time, the goal of the memory manager is to minimize the number of page faults. • When the size of a process' memory image increases (due to either stack expansion or data expansion), a new piece of memory big enough for the whole image is allocated. • If no single piece of main memory is large enough, the process is swapped out such that it will be swapped back in with the new size. 13
  • 14. • Decisions regarding which processes to swap in or swap out are made by the scheduler process (also known as the swapper). • A process is more likely to be swapped out if it is idle or has been in main memory for a long time, or is large ; if no obvious candidates are found, other processes are picked by age. • A process is more likely to be swapped in if its has been swapped out a long time, or is small. 14
  • 15. 15
  • 16. Allocating Swap Space Address Unit Allocate 100 unit 1 10000 101 9900 Map Allocate 50 unit Allocate 100 unit 251 9850 151 9750 16
  • 17. Freeing Swap Space Address Unit 50 unit free at 101 251 9750 101 50 Map 251 9750 Case 1: Free resources fill a hole, but not contiguous to any resources in the map 17
  • 18. Freeing Swap Space Address Unit 50 unit free at 101 101 50 251 9750 251 9750 Map 100 unit free at 1 1 150 Case 2: Free resources fill a hole, 251 9750 and immediately precedes an entry in the map 18
  • 19. Freeing Swap Space Address Unit 101 50 50 unit free at 101 251 9750 251 9750 Map 100 unit free at 1 1 150 Allocate 200 unit 1 150 451 9550 251 9750 300 unit free at 151 Case 3: Free resources fill a hole, and completely fills the 1 10000 gap between entries in the map 19
  • 20. Demand Paging • Demand paging to unix with BSD(Berkley system) which transferred memory pages instead of process to and from a secondary device. • When a process needs a page and the page is not there, a page fault to the kernel occurs a frame of main memory is allocated, and the process is loaded into the frame by the kernel. 20
  • 21. Page Table Page Frame Age Copy on Modify Reference Valid Protect Number write • frame # contains the physical frame where the virtual page is stored • age is processor dependant, and is meant to maintain how long it has been since the page was accessed. • Copy on Write store the copy on write bit, which is used in UNIX systems to, among other things, render fork efficient. 21
  • 22. • Dirty is a single bit that indicates whether a page has been modified since last swapped in (the opposite of dirty is clean, and a clean page need not be written out to disk if swapped). • Ref contains the usage information necessary for a CLOCK- style algorithm. • Valid is the standard UNIX jargon for resident. A valid page is in main memory, an invalid one is swapped out. • Protect contains the permission information for the page and is also hardware dependant. 22
  • 23. Disk block descriptor Swap device Number Device Block No. Type of storage • The disk block descriptor contains the information mapping a virtual page to a spot on disk. • The OS maintains a table of descriptors for each process. 23
  • 24. • Device # is basically a pointer to the disk that this page was swapped to. • Block # is the actual block that the page is stored on. This is why most UNIX systems prefer to have a separate swap partition, so that the block size can be set to the page size. • Type specifies whether the page is new or pre-existing. This lets the OS know if it has to clear the frame first. 24
  • 25. Page frame data table Page State Reference Logical Block Pf data Count device number pointer • The page frame data table holds information about each physical frame of memory (indexed by frame number). • This table is of primary importance for the replacement algorithm. 25
  • 26. • Page state indicates whether or not the frame is available or has an associated page (i.e. whether its been allocated to a process or not). • Ref. Count holds the number of processes that refer to this page (remember, processes can share the same physical page). • Logical device contains the device number of the disk that holds a physical copy of the page. 26
  • 27. • Block # holds the block number on that disk where the page data is located. • Pfdata pointer is a pointer field that is used to thread a singly-linked list of free frames through the frame table. If the page is free, this points to the next free page (useful for free list-style allocation). 27
  • 28. Swap use table Reference Page/storage count unit number • Reference Count : Number of page table entries that point to a page on the swap device. • Page/storage unit number : Page identifier on storage unit 28
  • 29. Page Replacement Algorithm • The Page frame data table is used for page replacement. • All of the available frames are linked together in a list of free frames available for bringing in pages. • The two-handed clock algorithm uses the reference bit in the page table entry for each page in memory that is eligible (not locked) to be swapped out. • This bit is set to 0 when when the page is first brought in and set to 1 when the page is refernced for a read or write . 29
  • 30. Two-Handed clock Page Replacement Algorithm 30
  • 31. • Two parameters determine the operation of the algorithm: – Scanrate: The rate at which the two hands scan through the page list, in pages per second. – Handspread: The gap between fronthand and backhand • These two parameters have default values set at boot time based on the amount of physical memory. 31
  • 32. • The parameter varies linearly between the values slowscan and fastscan as the amount of free memory varies between the values lotsfree and minfree. • The handspread parameter determines the gap between the fronthand and the backhand and therefore, together with scanrate,determines the window of oppurtunity to use a page before it is swapped out due to lack of use. 32
  • 33. Kernal Memory Allocator • Kernel memory allocator: provides buffers of memory to various kernel subsytems. Evolution Criteria : – must be space-efficient i.e. minimize wastage. – Can be measured by utilization factor – must be fast – must have a simple programming interface – should not force to free the entire allocated area all at once – must guard against the wastage of memory – must be able to interact with the paging system 33
  • 34. Advantages • easy to implement • not restricted to memory allocation • no wastage of space • can release any part of the region • allows reuse of memory by coalescing 34
  • 35. References Books : ********* 1. Os Internals & design --- William Stallings 2. The design of the unix operating system --- Maurice J. Bach prentice hall 35