Understanding the Xbox boot process/Flash structures

@xbox

Published in 

xbox

 · 9 months ago

written by Paul Bartholomew, 14 July 2002

This document will describe the Xbox's boot process, from RESET vector to KERNEL execution. In the process, the data structures/encodings used by the Xbox will be described.

It is not the intent of this document to describe methods of breaking any of the Xbox copy-protection mechanisms. I am a software developer, and respect the rights of copyright holders. Furthermore, there are already several 'mod chip' BIOS's available on the net - someone interested in bypassing copy-protection mechanisms would be using one of them - and would most likely not be interested in the details presented in this document.

The primary reason for this document is for the purpose of writing interoperable software under Sect. 1201 (f) Reverse Engineering exception of the DMCA.

Xbox boot Flash ROM

The standard retail Xbox uses a 1-Megabyte Flash ROM as the boot device. The Flash contains a bootloader (known as 'bootloader2' or '2bl') which will decrypt/decompress a KERNEL image (also stored in Flash) and execute it. The KERNEL contains hardware initialization code, and low-level hardware-access functions used by Xbox applications.

The Xbox's MCPX chip decodes the top 16-Megabytes of the CPU's address space (F0000000-FFFFFFFF) as the boot ROM memory region. Because the Flash chip is only 1-MByte, its contents will "mirror" 16-times throughout this region.

The standard retail Xbox's 1-Mbyte Flash chip actually contains 4 (identical) 256-Kbyte images (we'll call this image the "Xbox OS image"). So, the same 256-Kbyte image is repeated 64 times throughout the 16-MByte Flash memory region.

When making modifications to the Xbox Flash image, it is important to keep in mind that some data structures are accessed relative to the top of the chip, and others from the bottom of the chip (different 256-KByte regions of the 1-MByte Flash). It's easiest to just start with a 256-KByte image, make your modifications to that, then replicate it throughout the Flash chip.

Xbox OS image layout

Below is the basic layout of the Xbox OS image:

		+-------------------+ (top of image) 
		|      "Decoy"      | 
		|    Boot sector    | 
		+-------------------+ 
		|   Encrypted 2BL   | 
		|   (bootloader2)   | 
		+-------------------+ 
		|       KERNEL      | 
		|    initialized    | 
		|    data segment   | 
		|  (not encrypted)  | 
		+-------------------+ 
		|                   | 
		|                   | 
		|    Encrypted /    | 
		| compressed KERNEL | 
		|                   | 
		|                   | 
		+===================+ 
		|                   | 
		|Unknown3 - believed| 
		|    to be unused   | 
		|                   | 
		+===================+ 
		|    MS copyright   | 
		+-------------------+ 
		|     Unknown2      | 
		+-------------------+ 
		|      X-code       | 
		+-------------------+ 
		|Unknown1 - believed| 
		|  to be MCPX init  | 
		+-------------------+ (bottom of image (offset 0))

Using the standard (unmodified) retail Xbox kernel 3944, the locations of these structures within the image are as follows:

(B) denotes items that are addressed from the 'bottom' of the image (offset 0), (T) denotes items that are addressed from the 'top' of the image.

The Xbox boot process

MCPX boot sector functionality

At RESET time, it is believed that some hardware chipset initialization is done by the MCPX (by reading some data from the first 000080 bytes of Flash (CPU addresses F0000000-F000007F - based 16-MBytes below top of memory)). Once this is complete, CPU execution begins at the RESET vector (CPU address FFFFFFF0) in the MCPX 'boot sector'.

The MCPX boot sector is a small ROM image that's 'hidden' inside the MCPX chip. It replaces the top 000200-bytes of the address space, overlaying the "Decoy" boot sector in the Flash image.

The MCPX boot sector code does some basic hardware initialization/configuration by reading 'X-codes' from the X-code area starting at offset 000080 in the Flash (CPU address base FF000080 - note that this is approximately 1-MByte from the top of memory, not the full 16-Mbytes from top that the MCPX chipset allows). X-code processing has been fully documented in other places, so I won't repeat it here.

Once X-code processing is finished, the boot sector code decrypts the 'bootloader2' (2BL) image into RAM at CPU addresses 090000-095FFF. A standard RC4 decryption is used, with a 16-byte key stored at CPU address FFFFFFA5-FFFFFFB4 (inside the 'hidden' MCPX boot sector). After decryption, it verifies a 'magic number' at offset 005FE4 in the decrypted 2BL image is equal to 7854794A. If so, fetches a DWORD stored at CPU address 090000 (offset 000000 into decrpyted 2BL) and jumps to that address.

2BL functionality

The 2BL (bootloader2/secondary boot loader) is responsible for decrypting/decompressing the main KERNEL image and 'jumping' to it.

When 2BL starts, it is executing in a CPU address region from 090000-095FFF. It first sets up some simple page-tables (with what appears to be a one-to-one mapping of virtual to physical addresses), copies itself to CPU address region 400000-405FFF, enables paging, then 'jumps' to the copy of 2BL that it created based at 400000.

Next, the MCPX internal boot sector is 'hidden' (since it is no longer needed), and the PIC 'watchdog reset' is disabled (without doing this, the PIC chip will force a CPU reset after approximately 200ms of execution). The original decrypted copy of 2BL at CPU addresses 090000-095FFF is erased.

Some unknown initialization of video registers (memory-mapped based at CPU address FD000000) is done next, followed by some unknown PCI initialization.

Now for the 'guts' of 2BL: validation/decryption/decompression of the KERNEL.

The encrypted/compressed KERNEL image is located in Flash, "below" the KERNEL initialized data segment, which is located just "below" the encrypted 2BL (which starts at CPU address FFFF9E00). The size of the compressed KERNEL image is stored at offset 005FD8 into the 2BL, and the size of the KERNEL initialized data segment is stored at offset 005FDC into the 2BL. Using this information, the 2BL can find the start address/size of the encrypted/compressed KERNEL image.

First, a SHA-1 hash validation is done on the encrypted KERNEL image. The hash also includes some other items, like the RC4 key used to encrypt/decrypt the KERNEL, the unencrypted KERNEL initialized data segment, and the beginning of the Flash image up-to/including the MS copyright message (MCPX initialization, X-code, etc). The hash is compared against a 20-byte stored digest at offset 005FEC-005FFF into the decrypted 2BL image.

Next (assuming SHA-1 has was valid), the KERNEL image is decrypted to a temporary RAM buffer using an RC4 key stored at offset 00008C-00009B into the decrypted 2BL image. Note that this is not the same RC4 key that was used to decrypt the 2BL.

The KERNEL image is then decompressed to RAM starting at CPU address 80010000. The compression used for the KERNEL is a modified ".cab" (Microsoft CABinet) compression. See the Microsoft CABinet SDK for more detailed information (http://msdn.microsoft.com/library/en-us/dnsamples/cab-sdk.exe).

CAB compression allows for different types of compressions. The compression used for the KERNEL is "tcompTYPE_LZX" (Microsoft LZX). The CFHEADER, CFFOLDER, and CFFILE structures have been eliminated (since it's a single 'file') - only the CFDATA section is used. A slight modification to the CFDATA structure has been made: the 32-bit checksum ("u4 csum") stored at the start of each block has been eliminated. What remains are "cCFData" blocks of compressed data: each block starts with a 16-bit size of compressed data ("u2 cbData"), 16-bit size of uncompressed data ("u2 cbUncomp"), followed by a stream of "cbData" compressed data bytes.

The 2BL knows the value for "cCFDATA" (number of compressed CFDATA blocks) by adding-up the "u2 cbData" values from each CFDATA block header, until the total is equal to the total compressed KERNEL size (found at offset 005FD8 into the 2BL).

The decompressed kernel is a PE-format executable ('xboxkrnl.exe'). Once decompressed, the 2BL grabs the entry point address from the PE header and jumps to it. Two arguments are passed to the kernel entry point function: a pointer to string 'arguments' to the KERNEL (only used in debug kernel), and the base address of two 16-byte encryption keys. One of the keys is the EEPROM key (offset 00006C into 2BL), the other is the certificate key (offset 00007C into 2BL).

Kernel initialized data segment

The KERNEL initialized data segment has been mentioned several times in this document. This is simply a second copy of the section of the uncompressed KERNEL image that contains the initialized variables/etc.

When the KERNEL is running, and wants to re-initialize itself, it does a block-copy direct from the Flash KERNEL initialized data section into the corresponding RAM locations, and then zeros-out the BSS section.

Keeping this second copy uncompressed/unencrypted in Flash makes it easier/faster for the KERNEL to be able to re-initialized its data segment.

Making modifications to the KERNEL

It will be useful to be able to make small modifications to the KERNEL. An example would be in the case of trying to understand the Xbox hardware in order to port another OS to it (since the Xbox hardware isn't fully documented). Adding 'debug output' to the KERNEL at key places could help in understanding what certain code is doing.

The process for making a KERNEL modification would be:

• Decrypt 2BL (in order to get KERNEL decrypt key, size, other info)
• Decrypt/decompress KERNEL
• Make modifications to decompressed KERNEL image
• Make modifications to start of Flash image (X-codes, etc)
• Extract KERNEL initialized data segment from new KERNEL image
• Compress/encrypt new KERNEL image
• Re-calculate SHA-1 hash of KERNEL, initialized data segment, start of Flash image (X-codes, etc)
• Update 2BL with KERNEL info (size, hash, etc)
• Encrypt modified 2BL
• Reassemble all the pieces into a new Xbox OS image

Important data structure locations

Below are all of the locations of data structures needed in order to unpack/re-pack an Xbox OS image.

Standard algorithms used

The encryption/hashing functions are standard/fairly straightforward, and the code for the algorithms can be found on the net. I won't try to describe the details here.

It is necessary to know the specific use of the SHA-1 hashing functions in order to correctly generate a valid hash on a new image. Below is a code sample showing how this is done:

	// Initialize hash context 
	// 
	sha1_init(&sha1;_ctx); 

	// start with rc4 key used to encrypt KERNEL (@2BL+00008C) 
	// 
	sha1_update(&sha1;_ctx, KERNEL_RC4_key, 16); 

	// include size of compressed kernel image, then the compressed kernel 
	// image itself (size is value stored @2BL+005FE8) 
	// 
	sha1_update(&sha1;_ctx, (void *)&size;_kernelZ_image, 4); 
	sha1_update(&sha1;_ctx, p_kernelZ_image, size_kernelZ_image); 

	// include size of kernel initialized data image, then kernel 
	// initialized data image itself (size is value stored 
	// @2BL+005FDC and @KERNEL+00002C) 
	// 
	sha1_update(&sha1;_ctx, (void *)&kernel;_dataseg_size, 4); 
	sha1_update(&sha1;_ctx, p_kernel_dataseg, kernel_dataseg_size); 

	// include size of ROM-header/xcode image, then the image itself 
	// (size is value stored @2BL+005FE0) 
	// 
	sha1_update(&sha1;_ctx, size_flash_base_hash, 4); 
	sha1_update(&sha1;_ctx, p_flash_base, size_flash_base_hash); 

	sha1_final(sha1_digest, &sha1;_ctx); 

	// re-init 
	// 
	sha1_init(&sha1;_ctx); 

	// start with rc4 key used to encrypt KERNEL (@2BL+00008C) 
	// 
	sha1_update(&sha1;_ctx, KERNEL_RC4_key, 16); 

	// include digest computed above 
	// 
	sha1_update(&sha1;_ctx, sha1_digest, sizeof(sha1_digest)); 

	// Compute new digest - this should be stored @2BL+005FEC 
	// 
	sha1_final(sha1_digest, &sha1;_ctx);

Acknowledgements

A large amount of the information in this document was discovered by others, specifically those posting at www.xboxhacker.net forums. I have figured out some of this information myself, assisted others in their research, and put it all together into a single document to make the information more easily accessible.

I wouldn't know how to begin to list/give credit to all of the people involved - and I'd be sure to miss someone. I think that everyone in the Xbox community knows who these people are.