泰晓科技 -- 聚焦 Linux - 追本溯源,见微知著!
网站地址:https://tinylab.org

泰晓Linux知识星球:1300+知识点,520+用户
请稍侯

QEMU 启动方式分析(2): QEMU virt 平台下通过 OpenSBI + U-Boot 引导 RISCV64 Linux Kernel

ysyx 创作于 2022/11/23

Corrector: TinyCorrect v0.1-rc3 - [quotes header codeblock codeinline] Author: YJMSTR jay1273062855@outlook.com Date: 2022/08/23 Revisor: Bin Meng, Falcon Project: RISC-V Linux 内核剖析 Sponsor: PLCT Lab, ISCAS

使用软件版本如下:

QEMU: v7.0.0

OpenSBI: v1.1

U-Boot: v2022.04

Linux Kernel: v5.18

1 前言

在上一篇文章中,我们简要介绍了 RISC-V 的启动流程,并给出了在 QEMU RISCV ‘virt’ 平台下直接引导 Linux 内核以及使用 OpenSBI 引导 Linux 内核的步骤。本文将进一步在该平台下通过 U-Boot 与 OpenSBI 引导 Linux 内核,并简要介绍 QEMU 与 U-Boot 的相关命令与参数。

2 前期准备

这部分内容与 本系列上一篇文章 一致,不同之处在于制作根文件系统时需要将编译出的内核文件复制进来。

按如下命令下载并编译 OpenSBI:

  1. $ git clone https://gitee.com/tinylab/qemu-opensbi.git
  2. $ cd qemu-opensbi/
  3. $ export CROSS_COMPILE=riscv64-linux-gnu-
  4. $ make all PLATFORM=generic PLATFORM_RISCV_XLEN=64

按如下命令下载并编译 U-Boot:

  1. $ git clone https://gitee.com/mirrors/u-boot.git
  2. $ cd u-boot
  3. $ git checkout v2022.04
  4. $ export CROSS_COMPILE=riscv64-linux-gnu-
  5. $ make qemu-riscv64_smode_defconfig
  6. $ make -j $(nproc)

按如下命令编译内核:

  1. $ make ARCH=riscv CROSS_COMPILE=riscv64-linux-gnu- defconfig
  2. $ make ARCH=riscv CROSS_COMPILE=riscv64-linux-gnu- -j $(nproc)

使用 Busybox 制作根文件系统,将编译内核时编译出的 Kernel Image 也复制进来:

  1. $ git clone https://gitee.com/mirrors/busyboxsource
  2. $ cd busyboxsource
  3. $ export CROSS_COMPILE=riscv64-linux-gnu-
  4. $ make defconfig
  5. $ make menuconfig
  6. # 这里启用了 Settings-->Build Options 里的 Build static binary (no shared libs) 选项
  7. $ make -j $(nproc)
  8. $ make install
  9. $ cd ~
  10. $ qemu-img create rootfs.img 1g
  11. $ mkfs.ext4 rootfs.img
  12. $ mkdir rootfs
  13. $ sudo mount -o loop rootfs.img rootfs
  14. $ cd rootfs
  15. $ sudo cp -r ../busyboxsource/_install/* .
  16. $ sudo cp /path/to/Image .
  17. $ sudo mkdir proc sys dev etc etc/init.d
  18. $ cd etc/init.d/
  19. $ sudo touch rcS
  20. $ sudo vi rcS

编辑启动脚本 rcS 中的内容如下:

  1. #!/bin/sh
  2. mount -t proc none /proc
  3. mount -t sysfs none /sys
  4. /sbin/mdev -s

并修改文件权限:

  1. $ sudo chmod +x rcS
  2. $ cd ~
  3. $ sudo umount rootfs

3 引导内核

QEMU 启动命令:

  1. $ qemu-system-riscv64 -M virt -m 256M -nographic \
  2. -bios qemu-opensbi/build/platform/generic/firmware/fw_jump.bin \
  3. -kernel u-boot/u-boot-nodtb.bin \
  4. -drive file=rootfs.img,format=raw,id=hd0 \
  5. -device virtio-blk-device,drive=hd0 \
  6. -append "root=/dev/vda rw console=ttyS0"

最后在 U-Boot 中输入:

  1. => load virtio 0 0x80200000 \Image
  2. => booti 0x80200000 - $fdtcontroladdr

即可启动内核。

此处 load 命令的作用是从文件系统中读取一个二进制文件。在 U-Boot 中键入 help load,参考其帮助文档可知:命令 load virtio 0 0x80200000 \Image 表示从 virtio 类型的第 0 号设备的根目录中读取名为 Image 的文件到内存的 0x80200000 地址处。若我们在制作文件系统时进行了分区,则需要在设备号后通过 : 连接分区编号,以进一步指明从哪个分区中读取文件。由于我们需要加载整个 \Image 文件,故此处忽略 bytespos 参数。

booti 命令用于从内存中读取 Image 类型的内核文件。在 U-Boot 中键入 help booti,参考其帮助文档可知,命令 booti 0x80200000 - $fdtcontroladdr 的含义是引导存放于内存 0x80200000 处的内核文件,并加载位于 $fdtcontroladdr 处的设备树二进制文件(device tree blob,缩写为 dtb)。由于我们没有使用 initrd,而是使用了 /dev/sda,因此中间的 initrd[:size] 参数使用 - 忽略。

参数 fdt 用于指定 dtb 的地址,此处将其设置为了变量 fdtcontroladdr 的值。该变量表示 U-Boot 自身的 dtb 地址。

/path/to/u-boot/doc/usage/environment.rst 中,对其的描述如下:

fdtcontroladdr if set this is the address of the control flatteneddevice tree used by U-Boot when CONFIG_OF_CONTROL isdefined.

当启用了 CONFIG_OF_CONTROL 选项后,fdtcontroladdr 将作为 U-Boot 内置的设备树地址。在编译 U-Boot 时所选择的配置文件 qemu-riscv64_smode_defconfig 中,有这么一行代码:

  1. CONFIG_PREBOOT="setenv fdt_addr ${fdtcontroladdr}; fdt addr ${fdtcontroladdr};"

其将设备树地址设为 $fdtcontroladdr,并通过 fdt addr 命令将传递给操作系统的设备树地址设为 $fdtcontroladdrU-Boot 官方文档 中对于 fdt addr 的描述如下:

The working FDT is the one passed to the Operating System when booting. This can be freely modified, so far as U-Boot is concerned, since it does not affect U-Boot’s operation.

If the addr argument is provided, then this sets the address of the working or control FDT to the provided address.

在 U-Boot 中,除 booti 外,用于引导内核的命令还有:

  • bootm: 从内存中加载应用程序镜像。在使用其引导需要扁平设备树的 Linux 内核时,也需要提供 dtb 地址。可以用于引导 uImage 格式的 Linux 镜像。
  • bootz: 从内存中加载 zImage 格式的 Linux 镜像,参数格式同 booti
  • boot, bootd:按默认方式启动,例如通过 run 'bootcmd' 直接运行 bootcmd 变量对应的启动命令及参数。

U-Boot 还支持通过设置 bootargs 这一变量来向 Linux 内核传递启动参数(kernel command line)。在 QEMU 中通过 -append 这一启动参数也能实现这一功能。前文中是通过在 QEMU 启动命令中添加 -append "root=/dev/vda rw console=ttyS0" 进行内核启动参数的传递的,但这一参数必须要在有 -kernel 参数的情况下才能使用,因此有时我们需要在 U-Boot 中通过设置 bootargs 来传递启动命令。

最终的启动日志如下:

  1. OpenSBI v1.1
  2. ____ _____ ____ _____
  3. / __ \ / ____| _ \_ _|
  4. | | | |_ __ ___ _ __ | (___ | |_) || |
  5. | | | | '_ \ / _ \ '_ \ \___ \| _ < | |
  6. | |__| | |_) | __/ | | |____) | |_) || |_
  7. \____/| .__/ \___|_| |_|_____/|____/_____|
  8. | |
  9. |_|
  10. Platform Name : riscv-virtio,qemu
  11. Platform Features : medeleg
  12. Platform HART Count : 1
  13. Platform IPI Device : aclint-mswi
  14. Platform Timer Device : aclint-mtimer @ 10000000Hz
  15. Platform Console Device : uart8250
  16. Platform HSM Device : ---
  17. Platform Reboot Device : sifive_test
  18. Platform Shutdown Device : sifive_test
  19. Firmware Base : 0x80000000
  20. Firmware Size : 288 KB
  21. Runtime SBI Version : 1.0
  22. Domain0 Name : root
  23. Domain0 Boot HART : 0
  24. Domain0 HARTs : 0*
  25. Domain0 Region00 : 0x0000000002000000-0x000000000200ffff (I)
  26. Domain0 Region01 : 0x0000000080000000-0x000000008007ffff ()
  27. Domain0 Region02 : 0x0000000000000000-0xffffffffffffffff (R,W,X)
  28. Domain0 Next Address : 0x0000000080200000
  29. Domain0 Next Arg1 : 0x0000000082200000
  30. Domain0 Next Mode : S-mode
  31. Domain0 SysReset : yes
  32. Boot HART ID : 0
  33. Boot HART Domain : root
  34. Boot HART Priv Version : v1.10
  35. Boot HART Base ISA : rv64imafdch
  36. Boot HART ISA Extensions : time
  37. Boot HART PMP Count : 16
  38. Boot HART PMP Granularity : 4
  39. Boot HART PMP Address Bits: 54
  40. Boot HART MHPM Count : 0
  41. Boot HART MIDELEG : 0x0000000000001666
  42. Boot HART MEDELEG : 0x0000000000f0b509
  43. U-Boot 2022.04 (Aug 26 2022 - 11:49:17 +0800)
  44. CPU: rv64imafdcsuh
  45. Model: riscv-virtio,qemu
  46. DRAM: 256 MiB
  47. Core: 17 devices, 9 uclasses, devicetree: board
  48. Flash: 32 MiB
  49. Loading Environment from nowhere... OK
  50. In: uart@10000000
  51. Out: uart@10000000
  52. Err: uart@10000000
  53. Net: No ethernet found.
  54. Hit any key to stop autoboot: 0
  55. => load virtio 0 0x80200000 /Image
  56. 17685504 bytes read in 10 ms (1.6 GiB/s)
  57. => booti 0x80200000 - $fdtcontroladdr
  58. ## Flattened Device Tree blob at 8f73bad0
  59. Booting using the fdt blob at 0x8f73bad0
  60. Using Device Tree in place at 000000008f73bad0, end 000000008f73fdfd
  61. Starting kernel ...
  62. [ 0.000000] Linux version 5.18.0 (yjmstr@yjmstr) (riscv64-linux-gnu-gcc (Ubuntu 11.2.0-16ubuntu1) 11.2.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #2 SMP Sun Aug 14 13:14:26 CST 2022
  63. [ 0.000000] OF: fdt: Ignoring memory range 0x80000000 - 0x80200000
  64. [ 0.000000] Machine model: riscv-virtio,qemu
  65. [ 0.000000] efi: UEFI not found.
  66. [ 0.000000] Zone ranges:
  67. [ 0.000000] DMA32 [mem 0x0000000080200000-0x000000008fffffff]
  68. [ 0.000000] Normal empty
  69. [ 0.000000] Movable zone start for each node
  70. [ 0.000000] Early memory node ranges
  71. [ 0.000000] node 0: [mem 0x0000000080200000-0x000000008fffffff]
  72. [ 0.000000] Initmem setup node 0 [mem 0x0000000080200000-0x000000008fffffff]
  73. [ 0.000000] SBI specification v1.0 detected
  74. [ 0.000000] SBI implementation ID=0x1 Version=0x10001
  75. [ 0.000000] SBI TIME extension detected
  76. [ 0.000000] SBI IPI extension detected
  77. [ 0.000000] SBI RFENCE extension detected
  78. [ 0.000000] SBI SRST extension detected
  79. [ 0.000000] SBI HSM extension detected
  80. [ 0.000000] riscv: base ISA extensions acdfhim
  81. [ 0.000000] riscv: ELF capabilities acdfim
  82. [ 0.000000] percpu: Embedded 18 pages/cpu s34040 r8192 d31496 u73728
  83. [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 64135
  84. [ 0.000000] Kernel command line: root=/dev/vda rw console=ttyS0
  85. [ 0.000000] Dentry cache hash table entries: 32768 (order: 6, 262144 bytes, linear)
  86. [ 0.000000] Inode-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)
  87. [ 0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
  88. [ 0.000000] Virtual kernel memory layout:
  89. [ 0.000000] fixmap : 0xff1bfffffee00000 - 0xff1bffffff000000 (2048 kB)
  90. [ 0.000000] pci io : 0xff1bffffff000000 - 0xff1c000000000000 ( 16 MB)
  91. [ 0.000000] vmemmap : 0xff1c000000000000 - 0xff20000000000000 (1073741824 MB)
  92. [ 0.000000] vmalloc : 0xff20000000000000 - 0xff60000000000000 (17179869184 MB)
  93. [ 0.000000] lowmem : 0xff60000000000000 - 0xff6000000fe00000 ( 254 MB)
  94. [ 0.000000] kernel : 0xffffffff80000000 - 0xffffffffffffffff (2047 MB)
  95. [ 0.000000] Memory: 237564K/260096K available (6460K kernel code, 4865K rwdata, 2048K rodata, 2165K init, 334K bss, 22532K reserved, 0K cma-reserved)
  96. [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
  97. [ 0.000000] rcu: Hierarchical RCU implementation.
  98. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8 to nr_cpu_ids=1.
  99. [ 0.000000] rcu: RCU debug extended QS entry/exit.
  100. [ 0.000000] Tracing variant of Tasks RCU enabled.
  101. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
  102. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1
  103. [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
  104. [ 0.000000] riscv-intc: 64 local interrupts mapped
  105. [ 0.000000] plic: plic@c000000: mapped 53 interrupts with 1 handlers for 2 contexts.
  106. [ 0.000000] random: get_random_bytes called from start_kernel+0x4be/0x71a with crng_init=0
  107. [ 0.000000] riscv_timer_init_dt: Registering clocksource cpuid [0] hartid [0]
  108. [ 0.000000] clocksource: riscv_clocksource: mask: 0xffffffffffffffff max_cycles: 0x24e6a1710, max_idle_ns: 440795202120 ns
  109. [ 0.000110] sched_clock: 64 bits at 10MHz, resolution 100ns, wraps every 4398046511100ns
  110. [ 0.005388] Console: colour dummy device 80x25
  111. [ 0.010325] Calibrating delay loop (skipped), value calculated using timer frequency.. 20.00 BogoMIPS (lpj=40000)
  112. [ 0.010527] pid_max: default: 32768 minimum: 301
  113. [ 0.013571] Mount-cache hash table entries: 512 (order: 0, 4096 bytes, linear)
  114. [ 0.013624] Mountpoint-cache hash table entries: 512 (order: 0, 4096 bytes, linear)
  115. [ 0.044900] cblist_init_generic: Setting adjustable number of callback queues.
  116. [ 0.045100] cblist_init_generic: Setting shift to 0 and lim to 1.
  117. [ 0.045804] ASID allocator using 16 bits (65536 entries)
  118. [ 0.047123] rcu: Hierarchical SRCU implementation.
  119. [ 0.049057] EFI services will not be available.
  120. [ 0.051671] smp: Bringing up secondary CPUs ...
  121. [ 0.051788] smp: Brought up 1 node, 1 CPU
  122. [ 0.063866] devtmpfs: initialized
  123. [ 0.072712] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
  124. [ 0.073072] futex hash table entries: 256 (order: 2, 16384 bytes, linear)
  125. [ 0.079198] NET: Registered PF_NETLINK/PF_ROUTE protocol family
  126. [ 0.085551] cpuidle: using governor menu
  127. [ 0.116925] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
  128. [ 0.122252] iommu: Default domain type: Translated
  129. [ 0.122339] iommu: DMA domain TLB invalidation policy: strict mode
  130. [ 0.124185] SCSI subsystem initialized
  131. [ 0.126213] usbcore: registered new interface driver usbfs
  132. [ 0.126546] usbcore: registered new interface driver hub
  133. [ 0.126702] usbcore: registered new device driver usb
  134. [ 0.141528] vgaarb: loaded
  135. [ 0.143138] clocksource: Switched to clocksource riscv_clocksource
  136. [ 0.162394] NET: Registered PF_INET protocol family
  137. [ 0.164179] IP idents hash table entries: 4096 (order: 3, 32768 bytes, linear)
  138. [ 0.168491] tcp_listen_portaddr_hash hash table entries: 128 (order: 0, 5120 bytes, linear)
  139. [ 0.168769] TCP established hash table entries: 2048 (order: 2, 16384 bytes, linear)
  140. [ 0.169002] TCP bind hash table entries: 2048 (order: 4, 65536 bytes, linear)
  141. [ 0.169247] TCP: Hash tables configured (established 2048 bind 2048)
  142. [ 0.171587] UDP hash table entries: 256 (order: 2, 24576 bytes, linear)
  143. [ 0.171935] UDP-Lite hash table entries: 256 (order: 2, 24576 bytes, linear)
  144. [ 0.173301] NET: Registered PF_UNIX/PF_LOCAL protocol family
  145. [ 0.176516] RPC: Registered named UNIX socket transport module.
  146. [ 0.176585] RPC: Registered udp transport module.
  147. [ 0.176606] RPC: Registered tcp transport module.
  148. [ 0.176629] RPC: Registered tcp NFSv4.1 backchannel transport module.
  149. [ 0.176780] PCI: CLS 0 bytes, default 64
  150. [ 0.184118] workingset: timestamp_bits=62 max_order=16 bucket_order=0
  151. [ 0.200189] NFS: Registering the id_resolver key type
  152. [ 0.201294] Key type id_resolver registered
  153. [ 0.201351] Key type id_legacy registered
  154. [ 0.201767] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
  155. [ 0.201885] nfs4flexfilelayout_init: NFSv4 Flexfile Layout Driver Registering...
  156. [ 0.202545] 9p: Installing v9fs 9p2000 file system support
  157. [ 0.204311] NET: Registered PF_ALG protocol family
  158. [ 0.204862] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 250)
  159. [ 0.205046] io scheduler mq-deadline registered
  160. [ 0.205135] io scheduler kyber registered
  161. [ 0.217852] pci-host-generic 30000000.pci: host bridge /soc/pci@30000000 ranges:
  162. [ 0.218831] pci-host-generic 30000000.pci: IO 0x0003000000..0x000300ffff -> 0x0000000000
  163. [ 0.219443] pci-host-generic 30000000.pci: MEM 0x0040000000..0x007fffffff -> 0x0040000000
  164. [ 0.219525] pci-host-generic 30000000.pci: MEM 0x0400000000..0x07ffffffff -> 0x0400000000
  165. [ 0.220115] pci-host-generic 30000000.pci: Memory resource size exceeds max for 32 bits
  166. [ 0.221226] pci-host-generic 30000000.pci: ECAM at [mem 0x30000000-0x3fffffff] for [bus 00-ff]
  167. [ 0.223271] pci-host-generic 30000000.pci: PCI host bridge to bus 0000:00
  168. [ 0.223608] pci_bus 0000:00: root bus resource [bus 00-ff]
  169. [ 0.223724] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]
  170. [ 0.223807] pci_bus 0000:00: root bus resource [mem 0x40000000-0x7fffffff]
  171. [ 0.223825] pci_bus 0000:00: root bus resource [mem 0x400000000-0x7ffffffff]
  172. [ 0.225244] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000
  173. [ 0.305403] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
  174. [ 0.313925] printk: console [ttyS0] disabled
  175. [ 0.315969] 10000000.uart: ttyS0 at MMIO 0x10000000 (irq = 2, base_baud = 230400) is a 16550A
  176. [ 0.337906] printk: console [ttyS0] enabled
  177. [ 0.354794] loop: module loaded
  178. [ 0.359525] virtio_blk virtio0: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)
  179. [ 0.384530] e1000e: Intel(R) PRO/1000 Network Driver
  180. [ 0.384690] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
  181. [ 0.385178] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
  182. [ 0.385462] ehci-pci: EHCI PCI platform driver
  183. [ 0.385799] ehci-platform: EHCI generic platform driver
  184. [ 0.386088] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
  185. [ 0.386336] ohci-pci: OHCI PCI platform driver
  186. [ 0.386641] ohci-platform: OHCI generic platform driver
  187. [ 0.387906] usbcore: registered new interface driver uas
  188. [ 0.388252] usbcore: registered new interface driver usb-storage
  189. [ 0.389615] mousedev: PS/2 mouse device common for all mice
  190. [ 0.393187] goldfish_rtc 101000.rtc: registered as rtc0
  191. [ 0.393941] goldfish_rtc 101000.rtc: setting system clock to 2022-08-27T05:32:14 UTC (1661578334)
  192. [ 0.397014] syscon-poweroff soc:poweroff: pm_power_off already claimed for sbi_srst_power_off
  193. [ 0.397375] syscon-poweroff: probe of soc:poweroff failed with error -16
  194. [ 0.398866] sdhci: Secure Digital Host Controller Interface driver
  195. [ 0.399179] sdhci: Copyright(c) Pierre Ossman
  196. [ 0.399513] sdhci-pltfm: SDHCI platform and OF driver helper
  197. [ 0.401166] usbcore: registered new interface driver usbhid
  198. [ 0.401414] usbhid: USB HID core driver
  199. [ 0.401841] riscv-pmu-sbi: SBI PMU extension is available
  200. [ 0.402450] riscv-pmu-sbi: 16 firmware and 2 hardware counters
  201. [ 0.402646] riscv-pmu-sbi: Perf sampling/filtering is not supported as sscof extension is not available
  202. [ 0.407458] NET: Registered PF_INET6 protocol family
  203. [ 0.414490] Segment Routing with IPv6
  204. [ 0.414785] In-situ OAM (IOAM) with IPv6
  205. [ 0.415417] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
  206. [ 0.418568] NET: Registered PF_PACKET protocol family
  207. [ 0.420120] 9pnet: Installing 9P2000 support
  208. [ 0.420563] Key type dns_resolver registered
  209. [ 0.422637] debug_vm_pgtable: [debug_vm_pgtable ]: Validating architecture page table helpers
  210. [ 0.490329] EXT4-fs (vda): mounted filesystem with ordered data mode. Quota mode: disabled.
  211. [ 0.490900] VFS: Mounted root (ext4 filesystem) on device 254:0.
  212. [ 0.493945] devtmpfs: mounted
  213. [ 0.523717] Freeing unused kernel image (initmem) memory: 2164K
  214. [ 0.524690] Run /sbin/init as init process
  215. Please press Enter to activate this console.

4 小结

本文介绍了 QEMU RISCV64 ‘virt’ 平台下通过 OpenSBI + U-Boot 引导 Linux Kernel 的流程 以及 U-Boot 和 QEMU 的部分相关命令与参数。后续文章中将结合 QEMU 代码进一步分析 ZSBL 的行为以及 QEMU 的不同启动参数组合。

5 参考资料

  1. QEMU 6.1.0 运行 RISCV64 OpenSBI + U-Boot + Linux
  2. QEMU 运行 RISCV64 Linux
  3. QEMU 启动方式分析(1):QEMU 及 RISC-V 启动流程简介
  4. U-Boot 官方文档:fdt command


Read Album:

Read Related:

Read Latest: