目标
本文档主要的目的是要揭下磁盘存储逻辑结构的神秘面纱,与此同时,我们也介绍一些常见工具和数据结构。
我们讨论里涉及的操作,如果你有Linux虚拟机可用,则尽可以大胆尝试。对于透彻理解我们讨论的主题,亲自动手尝试是必须的。
尝试文档里的操作, 只需要复制粘贴。Enjoy youself.
必备条件
讨论中我们会直接使用诸如柱面、扇区、块等概念,如果对磁盘大致结构缺乏了解,请参考磁碟的组成复习。同样,我们也会直接使用两个工具dd和od,快速了解这两个工具可参考其他常见的压缩与备份工具:dd和非纯文字档: od。
除此之外,我们还会用到fdisk和parted工具。快速了解这两个工具的使用和用法,可以参考学习 Linux,101: 硬盘布局。
因为fdisk和parted的输出里会涉及到八进制和十六进制,所以,我们也会使用[bc]()工具。
最后一个工具是二进制编辑器。有使用顺手的,就直接使用好了。如果没有,可以从yum仓库搜索安装一个,或者参考Need a good hex editor for Linux,自行选择一款。
想必大家对大端序和小端序都有耳闻,对于一个字节内位的次序多少意识到过。我们要从二进制层面抽取存储信息,请大家注意对提取数据组合时的顺序。如果对这些名称生疏,请参考字节顺序。
对于操作二进制数据,Erlang内置的位串匹配操作无疑是最合适的。我们讨论中不会涉及之,但建议有兴趣的可以了解一下。
磁盘分区
磁盘有两种分区方式:MBR和GPT。因为ECS里默认使用的和经常见到都是MBR格式,所以这里我们只讨论MBR。
MBR
英文维基对MBR有全面、细致的介绍,中文维基对MBR的介绍也不错。建议参考这两篇文档。这里我们简略介绍之。
不想看源码的请直接参考维基链接。
MBR(Master Boot Record)是指磁盘第一块扇区上一种数据结构。在grub2里,被定义为
/* The structure of MBR. */
struct grub_msdos_partition_mbr
{
/* The code area (actually, including BPB). */
grub_uint8_t code[446];
/* Four partition entries. */
struct grub_msdos_partition_entry entries[4];
/* The signature 0xaa55. */
grub_uint16_t signature;
} GRUB_PACKED;
磁盘分区
磁盘分区数据是MBR的一部分,在grub里被定义为
/* The partition entry. */
struct grub_msdos_partition_entry
{
/* If active, 0x80, otherwise, 0x00. */
grub_uint8_t flag;
/* The head of the start. */
grub_uint8_t start_head;
/* (S | ((C >> 2) & 0xC0)) where S is the sector of the start and C
is the cylinder of the start. Note that S is counted from one. */
grub_uint8_t start_sector;
/* (C & 0xFF) where C is the cylinder of the start. */
grub_uint8_t start_cylinder;
/* The partition type. */
grub_uint8_t type;
/* The end versions of start_head, start_sector and start_cylinder,
respectively. */
grub_uint8_t end_head;
grub_uint8_t end_sector;
grub_uint8_t end_cylinder;
/* The start sector. Note that this is counted from zero. */
grub_uint32_t start;
/* The length in sector units. */
grub_uint32_t length;
} GRUB_PACKED;
而fdisk里则定义为
struct dos_partition {
unsigned char boot_ind; /* 0x80 - active */
unsigned char bh, bs, bc; /* begin CHS */
unsigned char sys_ind;
unsigned char eh, es, ec; /* end CHS */
unsigned char start_sect[4];
unsigned char nr_sects[4];
} __attribute__((packed));
Linux对分区数据的解读要比这个数据结构显示的复杂的多,细节请参考维基百科。我们讨论中只涉及分区数据里的type、start和length,即分区的类型、初始扇区和此分区的长度也即分区包含的扇区的总数量。
附注
以上两个数据结构都出自include/grub/msdos_partition.h文件。
初识MBR
如何提取MBR
我们可以使用下面的命令在终端里直接观察MBR
dd if=path_to_your_disc bs=1 count=512 2>/dev/null | od -v -tx1
也可以把MBR直接复制到一个文件里
dd if=path_to_your_disc bs=1 count=512 > path_to_mbr.bin
我们尝试一下
二进制编辑器里的MBR
87654321 0011 2233 4455 6677 8899 aabb ccdd eeff 0123456789abcdef
00000000: eb63 9010 8ed0 bc00 b0b8 0000 8ed8 8ec0 .c..............
00000010: fbbe 007c bf00 06b9 0002 f3a4 ea21 0600 ...|.........!..
00000020: 00be be07 3804 750b 83c6 1081 fefe 0775 ....8.u........u
00000030: f3eb 16b4 02b0 01bb 007c b280 8a74 018b .........|...t..
00000040: 4c02 cd13 ea00 7c00 00eb fe00 0000 0000 L.....|.........
00000050: 0000 0000 0000 0000 0000 0080 0100 0000 ................
00000060: 0000 0000 fffa 9090 f6c2 8074 05f6 c270 ...........t...p
00000070: 7402 b280 ea79 7c00 0031 c08e d88e d0bc t....y|..1......
00000080: 0020 fba0 647c 3cff 7402 88c2 52be 057c . ..d|<.t...R..|
00000090: b441 bbaa 55cd 135a 5272 3d81 fb55 aa75 .A..U..ZRr=..U.u
000000a0: 3783 e101 7432 31c0 8944 0440 8844 ff89 7...t21..D.@.D..
000000b0: 4402 c704 1000 668b 1e5c 7c66 895c 0866 D.....f..\|f.\.f
000000c0: 8b1e 607c 6689 5c0c c744 0600 70b4 42cd ..`|f.\..D..p.B.
000000d0: 1372 05bb 0070 eb76 b408 cd13 730d 5a84 .r...p.v....s.Z.
000000e0: d20f 83de 00be 857d e982 0066 0fb6 c688 .......}...f....
000000f0: 64ff 4066 8944 040f b6d1 c1e2 0288 e888 d.@f.D..........
00000100: f440 8944 080f b6c2 c0e8 0266 8904 66a1 .@.D.......f..f.
00000110: 607c 6609 c075 4e66 a15c 7c66 31d2 66f7 `|f..uNf.\|f1.f.
00000120: 3488 d131 d266 f774 043b 4408 7d37 fec1 4..1.f.t.;D.}7..
00000130: 88c5 30c0 c1e8 0208 c188 d05a 88c6 bb00 ..0........Z....
00000140: 708e c331 dbb8 0102 cd13 721e 8cc3 601e p..1......r...`.
00000150: b900 018e db31 f6bf 0080 8ec6 fcf3 a51f .....1..........
00000160: 61ff 265a 7cbe 807d eb03 be8f 7de8 3400 a.&Z|..}....}.4.
00000170: be94 7de8 2e00 cd18 ebfe 4752 5542 2000 ..}.......GRUB .
00000180: 4765 6f6d 0048 6172 6420 4469 736b 0052 Geom.Hard Disk.R
00000190: 6561 6400 2045 7272 6f72 0d0a 00bb 0100 ead. Error......
000001a0: b40e cd10 ac3c 0075 f4c3 0000 0000 0000 .....<.u........
000001b0: 0000 0000 0000 0000 8ee1 0e00 0000 8020 ...............
000001c0: 2100 83aa 2882 0008 0000 0000 2000 00aa !...(....... ...
000001d0: 2982 8efe ffff 0008 2000 00f8 df04 0000 )....... .......
000001e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000001f0: 0000 0000 0000 0000 0000 0000 0000 55aa ..............U.
初识分区
提取分区数据
分区数据是MBR的一部分,所以,只要调整一下取出MBR的命令,我们就能拿到分区数据
我们可以使用下面的命令在终端里直接观察分区数据
dd if=path_to_your_disc bs=1 count=64 skip=446 2>/dev/null | od -v -tx1
也可以把分区直接复制到一个文件里
dd if=path_to_your_disc bs=1 count=64 skip=446 > path_to_mbr.bin
我们测试一下命令,并把fdisk显示的一个分区与磁盘记录的数据联系起来。
分区类型
fdisk或者parted工具会显示分区类型,比如,如果分区类型字段83,如上例所示,则分区是Linux分区。那么,这个分区类型数据哪里可查呢?可以参考Partition type。fdisk源码里也给出了如下定义
{0x00, N_("Empty")},
{0x01, N_("FAT12")},
{0x02, N_("XENIX root")},
{0x03, N_("XENIX usr")},
{0x04, N_("FAT16 <32M")},
{0x05, N_("Extended")}, /* DOS 3.3+ extended partition */
{0x06, N_("FAT16")}, /* DOS 16-bit >=32M */
{0x07, N_("HPFS/NTFS/exFAT")}, /* OS/2 IFS, eg, HPFS or NTFS or QNX or exFAT */
{0x08, N_("AIX")}, /* AIX boot (AIX -- PS/2 port) or SplitDrive */
{0x09, N_("AIX bootable")}, /* AIX data or Coherent */
{0x0a, N_("OS/2 Boot Manager")},/* OS/2 Boot Manager */
{0x0b, N_("W95 FAT32")},
{0x0c, N_("W95 FAT32 (LBA)")},/* LBA really is `Extended Int 13h' */
{0x0e, N_("W95 FAT16 (LBA)")},
{0x0f, N_("W95 Ext'd (LBA)")},
{0x10, N_("OPUS")},
{0x11, N_("Hidden FAT12")},
{0x12, N_("Compaq diagnostics")},
{0x14, N_("Hidden FAT16 <32M")},
{0x16, N_("Hidden FAT16")},
{0x17, N_("Hidden HPFS/NTFS")},
{0x18, N_("AST SmartSleep")},
{0x1b, N_("Hidden W95 FAT32")},
{0x1c, N_("Hidden W95 FAT32 (LBA)")},
{0x1e, N_("Hidden W95 FAT16 (LBA)")},
{0x24, N_("NEC DOS")},
{0x27, N_("Hidden NTFS WinRE")},
{0x39, N_("Plan 9")},
{0x3c, N_("PartitionMagic recovery")},
{0x40, N_("Venix 80286")},
{0x41, N_("PPC PReP Boot")},
{0x42, N_("SFS")},
{0x4d, N_("QNX4.x")},
{0x4e, N_("QNX4.x 2nd part")},
{0x4f, N_("QNX4.x 3rd part")},
{0x50, N_("OnTrack DM")},
{0x51, N_("OnTrack DM6 Aux1")}, /* (or Novell) */
{0x52, N_("CP/M")}, /* CP/M or Microport SysV/AT */
{0x53, N_("OnTrack DM6 Aux3")},
{0x54, N_("OnTrackDM6")},
{0x55, N_("EZ-Drive")},
{0x56, N_("Golden Bow")},
{0x5c, N_("Priam Edisk")},
{0x61, N_("SpeedStor")},
{0x63, N_("GNU HURD or SysV")}, /* GNU HURD or Mach or Sys V/386 (such as ISC UNIX) */
{0x64, N_("Novell Netware 286")},
{0x65, N_("Novell Netware 386")},
{0x70, N_("DiskSecure Multi-Boot")},
{0x75, N_("PC/IX")},
{0x80, N_("Old Minix")}, /* Minix 1.4a and earlier */
{0x81, N_("Minix / old Linux")},/* Minix 1.4b and later */
{0x82, N_("Linux swap / Solaris")},
{0x83, N_("Linux")},
{0x84, N_("OS/2 hidden or Intel hibernation")},/* OS/2 hidden C: drive,
hibernation type Microsoft APM
or hibernation Intel Rapid Start */
{0x85, N_("Linux extended")},
{0x86, N_("NTFS volume set")},
{0x87, N_("NTFS volume set")},
{0x88, N_("Linux plaintext")},
{0x8e, N_("Linux LVM")},
{0x93, N_("Amoeba")},
{0x94, N_("Amoeba BBT")}, /* (bad block table) */
{0x9f, N_("BSD/OS")}, /* BSDI */
{0xa0, N_("IBM Thinkpad hibernation")},
{0xa5, N_("FreeBSD")}, /* various BSD flavours */
{0xa6, N_("OpenBSD")},
{0xa7, N_("NeXTSTEP")},
{0xa8, N_("Darwin UFS")},
{0xa9, N_("NetBSD")},
{0xab, N_("Darwin boot")},
{0xaf, N_("HFS / HFS+")},
{0xb7, N_("BSDI fs")},
{0xb8, N_("BSDI swap")},
{0xbb, N_("Boot Wizard hidden")},
{0xbc, N_("Acronis FAT32 LBA")},/* hidden (+0xb0) Acronis Secure Zone (backup software) */
{0xbe, N_("Solaris boot")},
{0xbf, N_("Solaris")},
{0xc1, N_("DRDOS/sec (FAT-12)")},
{0xc4, N_("DRDOS/sec (FAT-16 < 32M)")},
{0xc6, N_("DRDOS/sec (FAT-16)")},
{0xc7, N_("Syrinx")},
{0xda, N_("Non-FS data")},
{0xdb, N_("CP/M / CTOS / ...")},/* CP/M or Concurrent CP/M or
Concurrent DOS or CTOS */
{0xde, N_("Dell Utility")}, /* Dell PowerEdge Server utilities */
{0xdf, N_("BootIt")}, /* BootIt EMBRM */
{0xe1, N_("DOS access")}, /* DOS access or SpeedStor 12-bit FAT
extended partition */
{0xe3, N_("DOS R/O")}, /* DOS R/O or SpeedStor */
{0xe4, N_("SpeedStor")}, /* SpeedStor 16-bit FAT extended
partition < 1024 cyl. */
{0xea, N_("Rufus alignment")}, /* Rufus extra partition for alignment */
{0xeb, N_("BeOS fs")},
{0xee, N_("GPT")}, /* Intel EFI GUID Partition Table */
{0xef, N_("EFI (FAT-12/16/32)")},/* Intel EFI System Partition */
{0xf0, N_("Linux/PA-RISC boot")},/* Linux/PA-RISC boot loader */
{0xf1, N_("SpeedStor")},
{0xf4, N_("SpeedStor")}, /* SpeedStor large partition */
{0xf2, N_("DOS secondary")}, /* DOS 3.3+ secondary */
{0xfb, N_("VMware VMFS")},
{0xfc, N_("VMware VMKCORE")}, /* VMware kernel dump partition */
{0xfd, N_("Linux raid autodetect")},/* New (2.2.x) raid partition with
autodetect using persistent
superblock */
{0xfe, N_("LANstep")}, /* SpeedStor >1024 cyl. or LANstep */
{0xff, N_("BBT")}, /* Xenix Bad Block Table */
{ 0, NULL }
分区为什么从编号2048的扇区开始?
从上面的截图可以看到,/dev/sdb1分区是从2048个扇区开始的。为什么fdisk要这么设置分区呢?
在Linux系统上,_/sys/block/_下有众多的数据可以使用,比如,查看磁盘和其逻辑、物理块的大小
了解了这些数据后,请参考Windows Vista 对于大扇区硬盘驱动器的支持解答我们的问题。
磁盘和分区
我们以扇区为磁盘和分区的度量单位,看一下一块磁盘所有分区的情况。fdisk工具新老版本之间默认使用的单位可能是柱面,也可能是扇区。为统一计,我们总是使用下面这样的执行方式
fdisk -l -u path_to_your_disc
直接从硬盘提取分区信息的方法照旧。我们尝试一下
下文
如果您对逻辑分区和LVM感兴趣,请参考磁盘逻辑结构浅谈-逻辑分区和LVM