You can use a variety of strategies to optimize an IPL.
Techniques for reducing IPL boot times include:
When booting from eMMC enable power off notification (added in eMMC version 4.5). When booting from eMMC a large amount (several hundred milliseconds) is spent waiting for the eMMC device to be ready before the IPL even starts.
Power off notification allows the SDMMC driver to notify the eMMC part before the board shuts down, allowing subsequent boots to be faster. To enable power notification verify that the eMMC part supports power off notification, then make sure that the SDMMC driver uses the pwroff_notify option.
When using NOR Flash, use optimized NOR timing values. By default, an IPL will use large (slow) timing values to ensure that they are sufficient for a broad range of boards and conditions. (The IPL may be reused across a board family, for instance). Setting the timing to the optimal values eliminates unneeded waiting and can reduce the time the IPL needs to complete its tasks.
Check the hardware manufacturer's documentation for the optimal timing values for your specific board.
For more information about optimizing your boot, see the Boot Optimization Guide.