FORM OF NONQUALIFIED STOCK OPTION AGREEMENT

EXHIBIT 10.4

FORM OF

NONQUALIFIED STOCK OPTION AGREEMENT

THIS AGREEMENT is made by and between The Colonial BancGroup, Inc., a corporation organized under the laws of Delaware (hereinafter referred to as “BancGroup”), and [Name] (hereinafter referred to as “Optionee”).

WHEREAS, Optionee is a valuable and trusted BancGroup (or a subsidiary of BancGroup), and BancGroup considers it desirable and in its best interest that Optionee be given an inducement to achieve certain objectives as set forth in The Colonial BancGroup, Inc. 2001 Long-Term Incentive Plan (the “Plan”), adopted by the Board on the 17th day of January, 2001, and ratified by the stockholders of BancGroup on the 18th day of April, 2001.

WHEREAS, BancGroup believes that the issuance to Optionee of an option to purchase Stock will serve as such an inducement.

NOW, THEREFORE, in consideration of the premises, it is agreed by and between the parties as follows:

BancGroup grants to Optionee the right, privilege, and option to purchase [Number of options] shares of Stock at the purchase price of $[Price] per share in the manner and subject to the conditions hereinafter provided and as provided in the Plan. The effective date of this option is [Grant Date] (which date is the date on which this option was granted by the Committee or such other date designated by the Committee).

Except as set forth herein, the aforesaid option may be exercised at any time, and from time to time, in whole or in part, until the termination thereof as provided in Section 4 below; provided, however, that the options granted herein shall vest, provided the Optionee on the anniversary dates set forth herein, and be exercisable, pursuant to the following vesting schedule: 20% on the first anniversary of the effective date of this option, 40% on the second anniversary of the effective date of this option, 60% on the third anniversary of the effective date of this option, 80% on the fourth anniversary of the effective date of this option, and 100% on the fifth anniversary of the effective date of this option.

The option shall be exercised by written notice directed to the Committee, at the principal place of business of BancGroup, accompanied by a certified or cashier’s check in payment of the option price for the number of shares specified and paid for. BancGroup shall make immediate delivery of such shares, provided that if any law or regulation requires BancGroup to take any action with respect to the shares specified in such notice before the issuance thereof, then the date of delivery of such shares shall be extended for the period necessary to take such action.

Except as herein otherwise stated, the option, to the extent not theretofore exercised, shall terminate upon the first to occur of the following dates:

(a) the expiration of three (3) months after the date on which Optionee’s BancGroup, or any of its subsidiaries, is terminated (except if such termination be by a reason of death or Disability);

- 2 -

(b) the expiration of twelve (12) months after the date on which Optionee’s BancGroup, or any of its subsidiaries, is terminated, if such termination be by reason of Optionee’s Disability;

(c) in the event of Optionee’s death while in the BancGroup, or any of its subsidiaries, or during the three-month period described in paragraph 4(a) hereof or during the one-year period described in paragraph 4(b) hereof, and before the option otherwise lapses, the option shall lapse one year after the date of the appointment of a personal representative for such deceased Optionee’s estate. Upon the Optionee’s death, any exercisable option granted hereunder may be exercised by the Optionee’s estate or anyone who has acquired the option pursuant to the laws of descent and distribution;

(d) the [Term Date] (being the expiration of ten (10) years from the effective date of this option).

Notwithstanding any provision herein to the contrary, if Optionee’s is terminated as a result of deliberate, willful, or gross misconduct as determined by the Board or the Committee, all rights under this option shall terminate and expire upon such termination.

This option is non-transferable by Optionee, except in the event of Optionee’s death as provided in Section 4(c) above, and during Optionee’s lifetime is exercisable only by Optionee. Optionee shall have no rights as a stockholder with respect to the option shares until payment of the option price and delivery to Optionee of such shares as herein provided.

-3-

This Agreement is subject to, and incorporates by reference, all terms of the Plan. A copy of the Summary and Section 10(a) Prospectus for the Plan has been furnished to the Optionee, the receipt of which the Optionee hereby acknowledges. All terms capitalized herein shall have the same meaning given to such terms in the Plan unless specified otherwise herein.

This option is not intended to be treated as an Incentive Stock Option as defined in Section 422 of the Internal Revenue Code of 1986, as amended.

This Agreement shall inure to the benefit of and be binding upon the parties hereto and their respective heirs, executors, administrators, successors and assigns.

-4-

IN WITNESS WHEREOF, the parties hereto have caused this Agreement to be executed.

-5-