CHANGE IN CONTROL AGREEMENT

Exhibit 10.4

CHANGE IN CONTROL AGREEMENT

AGREEMENT by and between Colonial BancGroup, Inc., a Delaware corporation (the “Company”), and                      (the “Employee”), dated as of the          day of                     , 20        .

WHEREAS, the Board of Directors of the Company (the “Board”) recognizes the possibility that a Change in Control (as hereinafter defined) of the Company could occur and that such an event could result in significant distraction of the Company’s key personnel because of the uncertainties inherent in such a situation; and

WHEREAS, the Board has determined that it is essential and in the best interest of the Company and its stockholders to be able to retain the services of the Employee notwithstanding the possibility of a Change in Control and to ensure the Employee’s continued dedication and efforts in such an event without undue concern for the Employee’s personal financial and employment security;

NOW, THEREFORE, in consideration of the respective agreements of the parties set forth herein, it is hereby agreed as follows:

1. Certain Definitions.

(a) “Accrued Compensation” shall mean the sum of: (i) the Employee’s annual base salary through the Termination Date, to the extent not theretofore paid, (ii) reimbursement (in accordance with the Company’s expense reimbursement policy) for reasonable and necessary business expenses incurred by the Employee on behalf of the Company prior to the Termination Date, (iii) Employee’s accrued and unused vacation pay (in accordance with the Company’s vacation policy) to the extent not theretofore paid, and (iv) bonuses and incentive compensation, prorated through the Termination Date, to which the Employee is entitled under the terms of applicable bonus or incentive plans or awards maintained by the Company.

(b) “Affiliate” shall mean any entity directly or indirectly, controlled by, controlling or under common control with the Company or any corporation or other entity acquiring, directly or indirectly, all or substantially all the assets and business of the Company, whether by operation of law or otherwise.

(c) “Base Amount” shall mean the Employee’s annual base salary at the rate in effect at the date hereof or, if greater, at any time thereafter, determined without regard to any salary reduction or deferred compensation elections made by the Employee.

(d) “Bonus Amount” shall mean the highest bonus or bonuses paid or payable to Employee under the Company’s Management Team Incentive Plan (or such similar

successor plan as may be adopted by the Company and in which the Employee is a participant) in respect of any of the three (3) full fiscal years ended prior to the Termination Date or, if greater, the three (3) full fiscal years ended prior to the Change in Control.

(e) “Cause” shall include, but is not limited to the Following:

(i) the willful and continued failure of the Employee to perform substantially the Employee’s reasonably assigned duties with the Company or any of its Affiliates (other than any such failure resulting from incapacity due to physical or mental illness), which failure continued for a period of at least thirty (30) days after a written demand for substantial performance, signed by a duly authorized officer of the Company, has been delivered to the Employee specifying the manner in which the Employee has failed substantially to perform, or

(ii) the Employee’s breach of fiduciary duty involving personal profit, commission of a felony or a crime involving fraud or moral turpitude, or material breach of any provision of this Agreement, or

(iii) the willful engaging by the Employee in illegal conduct or gross misconduct which is materially injurious to the Company.

Notwithstanding the foregoing, no termination of the Employee’s employment shall be for Cause until (i) there shall have been delivered to the Employee a copy of a written notice, signed by a duly authorized officer of the Company, indicating that the Employee was guilty of the conduct described in this Section 1(e) and specifying the particulars thereof in detail, and (ii) the Employee shall have been provided an opportunity to be heard in person by the Personnel and Compensation Committee of the Board.

For purposes of this provision, no act or failure to act, on the part of the Employee, shall be considered “willful” unless it is done, or omitted to be done, by the Employee in bad faith or without reasonable belief that the Employee’s action or omission was legal, proper, and in the best interests of the Company. Any act, or failure to act, based upon authority and directives given pursuant to a resolution duly adopted by the Board or upon the lawful instructions of a senior officer of the Company or based upon the advice of counsel for the Company shall be conclusively presumed to be done, or omitted to be done, by the Employee in good faith and in the best interests of the Company. Notwithstanding anything set forth in this Agreement to the contrary, no failure to perform by the Employee after a Notice of Termination is given to the Company by the Employee shall constitute Cause for the purposes of this Agreement.

2

(f) “Change in Control” shall mean any of the following events:

(i) the acquisition by any “Person” (as the term “person” is used for the purposes of Section 13(d) or 14(d) of the Securities Exchange Act of 1934, as amended (the “Exchange Act”)) of direct or indirect beneficial ownership (within the meaning of Rule 13d-3 promulgated under the Exchange Act) of 50% or more of the combined voting power of the then-outstanding securities of the Company entitled to vote in the election of directors (the “Voting Securities”); or

(ii) individuals who, as of the date hereof, constitute the Board (the “Incumbent Directors”) cease for any reason to constitute at least a majority of the Board; provided, however, that any individual becoming a director subsequent to the date hereof whose election, or nomination for election, was approved by a vote of at least a majority of the Incumbent Directors then on the Board, or the Nominating and Corporate Governance Committee of the Board, shall be an Incumbent Director, unless such individual is initially elected or nominated as a director of the Company as a result of an actual or threatened election contest with respect to the election or removal of directors (“Election Contest”) or other actual or threatened solicitation of proxies or consents by or on behalf of a Person other than the Board (“Proxy Contest”), including by reason of any agreement intended to avoid or settle any Election Contest or Proxy Contest; or

(iii) The consummation of a merger, consolidation, reorganization, statutory share exchange, or similar form of corporate transaction involving the Company, the sale or other disposition of all or substantially all of the Company’s assets, or the acquisition of assets or stock of another entity by the Company (each a “Business Combination”), unless such Business Combination is a “Non-Control Transaction.” A “Non-Control Transaction” is a Business Combination immediately following which the following conditions are met:

(A) the stockholders of the Company immediately before such Business Combination own, directly or indirectly, more than fifty percent (50%) of the combined voting power of the then-outstanding voting securities entitled to vote in the election of directors of the corporation resulting from such Business Combination (including, without limitation, a corporation that as a result of such Business Combination owns the Company or all of substantially all of the Company’s assets or stock either directly or through one or more subsidiaries) (the “Surviving Corporation”) in substantially the same proportion as their ownership of the Company Voting Securities immediately before such Business Combination;

(B) at least a majority of the members of the board of directors of the Surviving Corporation were Incumbent Directors at the time of the Board’s approval of the execution of the initial Business Combination agreement; and

3

(C) no person other than (i) the Company or any of its subsidiaries, (ii) the Surviving Corporation or its ultimate parent corporation, or (iii) any employee benefit plan (or related trust) sponsored or maintained by the Company immediately prior to such Business Combination beneficially owns, directly or indirectly, fifty percent (50%) or more of the combined voting power of the Surviving Corporation’s then-outstanding voting securities entitled to vote in the election of directors; or

(iv) Approval by the stockholders of the Company of a complete liquidation or dissolution of the Company.

Notwithstanding the foregoing, a Change in Control shall not be deemed to occur solely because any Person (the “Subject Person”) acquired Beneficial Ownership of more than the permitted amount of the outstanding Voting Securities as a result of the acquisition of Voting Securities by the Company which, by reducing the number of Voting Securities outstanding, increases the proportional number of shares Beneficially Owned by the Subject Person, provided that if a Change in Control would occur (but for the operation of this sentence) and after such acquisition of Voting Securities by the Company, the Subject Person becomes the Beneficial Owner of any additional Voting Securities, then a Change in Control shall occur.

(g) “Company” shall mean Colonial BancGroup, Inc. its successors and assigns.

(h) “Disability” shall mean that the Employee has become eligible to receive benefits under any group long-term disability plan or policy maintained by the Company or any of its Affiliates that is by its terms applicable to the Employee.

(i) “Effective Date” shall mean the first date on which a merger is consummated.

(j) “Good Reason” shall mean the occurrence, after a Change in Control, of any of the following events or conditions:

(i) a material adverse change in the Employee’s status, position or responsibilities as in effect immediately prior to such Change in Control, or the assignment to the Employee of any duties or responsibilities which are materially inconsistent with his status, title, position or responsibilities immediately prior to such Change in Control;

(ii) a reduction of the Employee’s annual base salary below the Base Amount;

4

(iii) the Company requiring the Employee to be based at any location that is more than fifty (50) miles from the Employee’s regular place of employment immediately prior to the Change in Control;

(iv) the failure by the Company to pay to the Employee any portion of the Employee’s current compensation, or to pay to the Employee any portion of an installment of deferred compensation under any deferred compensation program of the Company in which the Employee participated, within seven (7) days of the date such compensation is due;

(v) the failure by the Company to (i) continue in effect (without reduction in benefit level, and/or award opportunities) any material compensation or employee benefit plan in which the Employee was participating immediately prior to the Change in Control, unless a substitute or replacement plan has been implemented which provides substantially the same compensation or benefits to the Employee or (ii) provide the Employee with compensation and benefits, in the aggregate, at least equal (in terms of benefit levels and/or award opportunities) to those provided for under each compensation or other employee benefit plan, program and practice in which the Employee was participating immediately prior to the Change in Control, unless there is a universal reduction for all Employees;

(vi) any failure of the Company to comply with and satisfy Section 8(c) of this Agreement; or

Any event described in Section 1(j)(i) through (v) above which occurs or within six (6) months prior to a Change in Control and which (i) was at the request of a third party who has taken steps reasonably calculated to effect a Change in Control or (ii) otherwise arose in connection with or anticipation of a Change in Control, shall constitute Good Reason for purposes of this Agreement notwithstanding the fact that it occurred prior to a Change in Control.

(k) “Notice of Termination” shall mean written notice, following a Change in Control, of termination of the Employee’s employment signed by the Employee if to the Company or by a duly authorized officer of the Company if to the Employee, which indicates the specific termination provision in this Agreement, if any, relied upon and which sets forth in reasonable detail the facts and circumstances claimed to provide the basis for termination of the Employee’s employment under the provision so indicated.

(l) “Termination Date” shall mean (i) in the case of the Employee’s death, the date of the Employee’s death, (ii) if the Employee’s employment is terminated for Disability, thirty (30) days after Notice of Termination is given (provided that the Employee shall not have returned to the performance of the Employee’s duties on a full-time basis during such thirty (30) day period), and (iii) if the Employee’s employment is

5

terminated for any other reason, the date specified in the Notice of Termination (which, in the case of a termination for Cause shall not be less than thirty (30) days, and in the case of a termination for Good Reason shall not be more than sixty (60) days, from the date such Notice of Termination is given).

2. Term of Agreement. This Agreement shall commence as of the date hereof, and shall continue in effect for a term of two (2) years unless extended as hereinafter provided. On the second anniversary of this Agreement, and on each anniversary thereafter, the term of this Agreement shall automatically be extended for one additional year unless at least 30 days prior to any such anniversary, the Company shall give written notice to the Employee of its desire not to further extend the term of this Agreement. Upon receipt of such notice by the Employee, no further automatic extensions shall occur, and this Agreement will terminate at the end of the then-current term. This Agreement shall automatically terminate upon the termination of the Employee’s employment with the Company and its Affiliates under circumstances not involving a Change in Control.

3. Termination of Employment. If the Employee’s employment with the Company and with its Affiliates shall be terminated within twenty four (24) months following a Change in Control (or within six (6) months prior to a Change in Control if such termination (i) was at the request of a third party who has taken steps reasonably calculated to effect a Change in Control or (ii) otherwise arose in connection with or anticipation of a Change in Control), the Employee shall be entitled to the following compensation and benefits:

(a) If such termination of the Employee’s employment is (i) by the Company for Cause or Disability, (ii) by reason of the Employee’s death, or (iii) by the Employee other than for Good Reason, the Company shall pay to the Employee the Employee’s Accrued Compensation.

(b) If such termination of the Employee’s employment is (i) by the Company without Cause, or (ii) by the Employee for Good Reason, the Company shall pay to the Employee the aggregate of the Employee’s Accrued Compensation plus an amount equal to three (3) times the sum of the Employee’s Base Amount and Bonus Amount.

(c) The amounts provided for in Section 3(a) and 3(b) shall be paid in a single lump sum cash payment within thirty (30) days after the Employee’s Termination Date; provided that, if the Employee has elected a different payout date in a prior deferral election with respect to any Accrued Compensation consisting of compensation previously deferred by the Employee (together with any accrued interest or earnings thereon), such deferred amounts shall be paid pursuant to the terms of such election.

(d) The severance pay and benefits provided for in this Section 3 shall be in lieu of any other severance pay to which the Employee shall be entitled under the Company’s Severance Pay Plan or any other plan, agreement, or arrangement of the Company or any Affiliate. The Employee’s entitlement to any other benefits (other than additional severance pay) shall be determined in accordance with the Company’s employee benefit plans and other applicable programs and practices then in effect.

6

4. Excise Tax Limitation.

(a) Anything in this Agreement to the contrary notwithstanding, in the event that any amount or benefit paid or distributed to the Employee pursuant to this Agreement, taken together with any amounts or benefits otherwise paid or distributed to the Employee by the Company or any Affiliate under any other plan, agreement, or arrangement that would be taken into account for purposes of determining if an “excess parachute payment” as defined in Section 280G of the Internal Revenue Code of 1986, as amended, has been made (collectively, the “Covered Payments”), would be an “excess parachute payment” as defined in Section 280G of the Internal Revenue Code of 1986, as amended (the “Code) , and would thereby subject the Employee to the tax (the, “Excise Tax”) imposed under Section 4999 of the Code (or any similar tax that may hereafter be imposed), the Company shall pay to the Employee an additional amount (the “Tax Reimbursement Payment”) such that the net amount retained by the Employee with respect to such Covered Payments, after deduction of any Excise Tax on the Covered Payments and Excise Tax on the Tax Reimbursement Payment provided for by this Section 4(a), shall be equal to the aggregate value of the Covered Payments; provided, however, that if the aggregate value of all Covered Payments exceeds the maximum amount which can be paid to the Employee without the Employee incurring an Excise Tax (the “Cap Amount”) by less than ten per cent (10%) of the Cap Amount, the amounts payable to the Employee under this Section shall be reduced (but not below zero) to the maximum amount which may be paid hereunder without the Employee becoming subject to such an Excise Tax as a result of all Covered Payments (such reduced payments to be referred to as the “Payment Cap”). In the event that Employee receives reduced payments and benefits hereunder, the Employee shall have the right to designate which of the payments and benefits otherwise provided for in this Agreement that he/she will receive in connection with the application of the Payment Cap.

(b) All determinations required to be made under this Section 4, including whether an Excise Tax would otherwise be imposed and the assumptions to be utilized in arriving at such determination, shall be made by a nationally recognized accounting firm as designated by the Board (the “Accounting Firm”) which shall provide detailed supporting calculations both to the Company and the Employee within 15 business days of the receipt of Notice of Termination from the Employee, or such earlier time as is requested by the Company. In the event that the Accounting Firm is serving as accountant or auditor for the individual, entity or group effecting the Change in Control, the Board may appoint another nationally recognized accounting firm to make the determinations required hereunder (which accounting firm shall then be referred to as the Accounting Firm hereunder). All fees and expenses of the Accounting Firm shall be borne solely by the Company. Any determination by the Accounting Firm shall be

7

binding upon the Company and the Employee. As a result of the uncertainty in the application of Section 4999 of the Code at the time of the initial determination by the Accounting Firm hereunder, it is possible that Payments hereunder will have been unnecessarily limited by this Section 4 (“Underpayment”), consistent with the calculations required to be made hereunder. The Accounting Firm shall determine the amount of the Underpayment that has occurred, and any such Underpayment shall be promptly paid by the Company to or for the benefit of the Employee.

5. Nonexclusivity of Rights. Nothing in this Agreement shall prevent or limit the Employee’s continuing or future participation in any employee benefit plan, program, policy or practice provided by the Company or any of its Affiliates and for which the Employee may qualify, except as specifically provided herein. Amounts which are vested benefits or which the Employee is otherwise entitled to receive under any plan, policy, practice or program of or any contract or agreement with the Company or any of its Affiliates at or subsequent to the Termination Date shall be payable in accordance with such plan, policy, practice or program or contract or agreement except as explicitly modified by this Agreement.

6. Full Settlement. The Company’s obligation to make the payments provided for in this Agreement and otherwise to perform its obligations hereunder shall not be affected by any setoff, counterclaim, recoupment, defense or other claim, right or action which the Company may have against the Employee or others. In no event shall the Employee be obligated to seek other employment or take any other action by way of mitigation of the amounts payable to the Employee under any of the provisions of this Agreement and such amounts shall not be reduced whether or not the Employee obtains other employment.

7. Fees and Expenses. The Company agrees to pay, to the full extent permitted by law, all legal fees and expenses reasonably incurred by the Employee as a result of any contest by the Company, the Employee or others of the validity or enforceability of, or liability under, any provision of this Agreement or any guarantee thereof; provided, however, that the Company shall not be liable for any such fees or expenses if a court determines that the position taken by the Employee with respect to such contest is not a reasonable position or is frivolous, and the Company shall not be liable for further fees or expenses incurred in any such contest beyond the time at which a reasonable person would or should know that the Employee’s position is not reasonable or is frivolous.

8. Successors.

(a) This Agreement is personal to the Employee and without the prior written consent of the Company shall not be assignable by the Employee otherwise than by will or the laws of descent and distribution. This Agreement shall inure to the benefit of and be enforceable by the Employee’s legal representatives.

(b) This Agreement shall inure to the benefit of and be binding upon the Company and its successors and assigns.

8

(c) The Company will require any successor (whether direct or indirect, by purchase, merger, consolidation or otherwise) to all or substantially all of the business and/or assets of the Company to assume expressly and agree to perform this Agreement in the same manner and to the same extent that the Company would be required to perform it if no such succession had taken place. As used in this Agreement, “Company” shall mean the company as hereinbefore defined and any successor to its business and/or assets as aforesaid which assumes and agrees to perform this Agreement by operation of law, or otherwise.

9. Miscellaneous.

(a) This Agreement shall be governed by and construed in accordance with the laws of the State of Alabama, without reference to principles of conflict of laws. The captions of this Agreement are not part of the provisions hereof and shall have no force or effect. This Agreement may not be amended or modified otherwise than by a written agreement executed by the parties hereto or their respective successors and legal representatives.

(b) All notices and other communications hereunder shall be in writing and shall be given by hand delivery to the other party or by registered or certified mail, return receipt requested, postage prepaid, addressed as follows:

or to such other address as either party shall have furnished to the other in writing in accordance herewith. Notice and communications shall be effective when actually received by the addressee.

(c) The invalidity or unenforceability of any provision of this Agreement shall not affect the validity or enforceability of any other provision of this Agreement.

(d) The Company may withhold from any amounts payable under this Agreement such Federal, state, local or foreign taxes as shall be required to be withheld pursuant to any applicable law or regulation.

9

(e) The Employee’s or the Company’s failure to insist upon strict compliance with any provision of this Agreement or the failure to assert any right the Employee or the Company may have hereunder, including, without limitation, the right of the Employee to terminate employment for Good Reason, shall not be deemed to be a waiver of such provision or right or any other provision or right of this Agreement.

(f) The Employee and the Company acknowledge that, except as may otherwise be provided under any other written agreement between the Employee and the Company, the employment of the Employee by the Company is “at will” and, subject to Section 3 hereof, prior to the Effective Date, the Employee’s employment may be terminated by either the Employee or the Company at any time, in which case the Employee shall have no further rights under this Agreement.

(g) This agreement sets forth the entire agreement of the parties hereto and supersedes all prior agreements, understandings and covenants (except as otherwise provided herein) with respect to the subject matter hereof. This agreement may be amended or canceled only by mutual agreement of the parties and only in writing.

IN WITNESS WHEREOF, the Employee has hereunto set the Employee’s hand and, pursuant to the authorization from its Board of Directors, the Company has caused these presents to be executed in its name on its behalf, all as of the day and year first above written.

10