PROMISSORY NOTE

Exhibit 10.2

PROMISSORY NOTE

FOR VALUE RECEIVED, the undersigned BancTrust Financial Group, Inc., an Alabama corporation (the “Borrower”), hereby promises to pay to the order of The Bankers Bank, N.A., a national banking association (the “Bank”), at its office at 3800 Colonnade Parkway, Birmingham, Alabama ###-###-####, or at such other place as the Bank may direct, in lawful money of the United States of America constituting legal tender in payment of all debts and dues, public and private, together with interest thereon calculated at the rate and in the manner set forth in this Note, the principal amount of THIRTY-EIGHT MILLION DOLLARS ($38,000,000.00) (the “Maximum Outstanding Principal Balance”). Interest shall accrue and payments shall be made in accordance with that certain Loan Agreement between the Borrower and the Bank dated as of the date hereof (the “Loan Agreement” and, together with all other documents and instruments evidencing, securing, or otherwise relating to the indebtedness evidenced by this Note, the “Loan Documents”) and the following provisions (provided, however, that as to any inconsistency between the provisions of the Loan Agreement and the Note, the latter shall govern and control):

1. Definitions. The following terms, as used in this Note, shall have the following meanings. Capitalized terms used but not defined herein shall have the meaning given to the same in the Loan Agreement.

(a) “Business Day” shall mean any day, Monday through Friday, on which the Bank is open for the conduct of its general banking business.

(b) “Interest Payment Date” shall mean the 15th of January, April, July, and October until all outstanding amounts evidenced by this Note have been paid in full, commencing on the date hereof.

(c) “LIBOR Rate” shall mean the one-month LIBOR Rate, as announced in the “Money Rates” section of the Wall Street Journal. Each change in the LIBOR Rate shall become effective on the date the rate is published.

(d) “NPA Ratio” shall mean the Non-Performing Assets Ratio as defined in Section 4.03(a) of the Loan Agreement.

(e) “Required Ratio” shall mean the NPA Ratio that the Borrower is required by Section 4.03(a) of the Loan Agreement to satisfy at the times specified in such section.

2. Payment. The Borrower promises to pay accrued interest quarterly, on or before each Interest Payment Date. The first Interest Payment Date shall be January 15, 2008. The balance of the principal and all accrued and unpaid interest on this Note and all charges under this Note and the Loan Documents shall be due and payable on October 16, 2010.

1

3. Interest.

(a) Interest shall accrue on the outstanding principal balance and interest thereon at the applicable interest rate determined and calculated as set forth below.

(b) The applicable interest rate under this Note shall be determined by the Borrower’s NPA Ratio measured as of the end of each fiscal quarter and the Borrower shall pay such rate of interest based upon the NPA Ratio as set forth in the table below, which rate shall change and become effective upon the first day of the next succeeding month following a change in such ratio; provided, however, that the applicable interest rate under this Note shall not be more than the maximum rate allowed by applicable law.

(d) Interest on the outstanding principal balance and interest thereon shall be calculated, charged, and paid on the basis of a 360-day year applied to the actual number of days upon which the outstanding principal balance is outstanding, by multiplying the product of the outstanding principal balance and the applicable rate set forth herein by the actual number of days elapsed, and dividing by 360.

(e) The Bank uses various referencing rates or indices in establishing interest rates, including but not limited to the LIBOR Rate and the LIBOR Index Rate. The Borrower acknowledges that the Bank may lend to others at rates of interest at, or greater or less than, such indices or the rates provided in this Note.

2

5. Late Charge. The Borrower shall pay a late charge in an amount equal to 5% of any payment which is not received by the Bank within ten (10) days of the date on which the same is due and payable.

6. Maturity. The entire outstanding principal balance, interest, and all other charges under this Note and the other Loan Documents shall be due and payable in full on October 16, 2010, which shall be the maturity date of this Note.

7. Prepayment. If this Note is pre-paid partially or in its entirety before August 31, 2008, a pre-payment penalty equal to the product of the principal balance so prepaid multiplied by five basis points will be due and payable at the time of any such pre-payment. If the pre-payment occurs from the proceeds of a capital markets financing arranged, made or managed by BankersBanc Capital Corporation or its affiliates, no pre-payment penalty will be incurred. On or after August 31, 2008, this Note may be prepaid in whole or in part at any time without fee or penalty. In accordance with Section 7.05 of the Loan Agreement, if the Note is sold or transferred, the pre-payment penalty shall be waived by the Bank.

8. Indemnity. The Borrower hereby agrees to indemnify the Bank, its officers, employees, and agents from any cost or loss arising from their actions taken or omitted to be taken in good faith based upon communications between the Borrower and the Bank. The obligations of the Borrower under this Section shall survive payment of this Note.

9. Loan Documents. This Note is included in the indebtedness referred to in the Loan Documents and is entitled to the benefits of those documents, but neither this reference to those documents nor any provisions thereof shall affect or impair the absolute and unconditional obligations of the Borrower to pay the principal of and interest on this Note when due.

10. Events of Default. The occurrence of any one or more of the following events shall constitute an event of default under this Note (“Events of Default”):

(a) Default in the payment of the principal or interest on this Note as and when the same is due and payable; or

(c) The occurrence of any default or event of default specified in the Loan Agreement, the other Loan Documents, or in any other instrument executed in connection with or securing this Note.

Upon the occurrence of an Event of Default, or at any time thereafter during the continuance of any such event, and following the expiration of any applicable cure period, the holder, with or without notice to the Borrower, may declare this Note and indebtedness evidenced hereby to be forthwith due and payable, whereupon this Note and the indebtedness evidenced hereby shall become forthwith due and payable, both as to principal and interest, without presentment, demand, protest, or other notice of any kind, all of which are hereby expressly waived, anything contained in this Note or in any of the Loan Documents or in any other instrument executed in connection with this Note to the contrary notwithstanding. No failure of any holder of this Note

3

to accelerate the indebtedness evidenced hereby or to exercise any other right hereunder shall be construed as a novation or modification of this Note, or the obligations evidenced hereby, or a waiver of the holder’s right to thereafter insist upon strict compliance with the terms of this Note without prior notice of such intention being given to the Borrower.

11. Waivers. The Borrower hereby waives demand, presentment for payment, notice of dishonor, protest, and notice of protest and diligence in collection or bringing suit and agrees that the holder of this Note may accept partial payment or release, without discharging or releasing any unreleased collateral or the obligations evidenced by this Note. The Borrower further waives any and all rights of exemption, both as to personal and real property, under the constitution or laws of the United States, the State of Alabama, or any other state.

12. Attorneys’ Fees. The Borrower agrees to pay attorneys’ fees and costs incurred by the holder of this Note in collecting or attempting to collect this Note, whether by suit or otherwise.

13. Miscellaneous. As used herein, the terms “Borrower,” “Bank,” and “holder” shall be deemed to include their respective successors, legal representatives, and assigns, whether by voluntary action of the parties or by operation of law. This Note is given under the seal of all parties hereto, and it is intended that this Note is and shall constitute and have the effect of a sealed instrument according to law. This Note has been negotiated, and is being executed and delivered in Birmingham, in the State of Alabama, or if executed elsewhere, shall become effective upon the Bank’s receipt and acceptance of the executed original of this Note in the State of Alabama; provided, however, that the Bank shall have no obligation to give, nor shall the Borrower be entitled to receive, any notice of such acceptance for this Note to become a binding obligation of the Borrower. The Borrower hereby submits to jurisdiction in the State of Alabama. This Note shall be governed by and be construed in accordance with the laws of the State of Alabama. It is intended, and the Borrower and the holder of this Note specifically agree, that the laws of the State of Alabama governing interest shall apply to this Note and to this transaction. This Note may not be modified except by written agreement signed by the Borrower and the holder hereof, or by their respective successors or assigns.

IN WITNESS WHEREOF, Borrower has caused this Note to be executed, sealed and delivered as of the date first set forth above.

4

I, the undersigned, Notary Public in and for said County in said State, hereby certify that W. Bibb Lamar, Jr., whose name as President of BancTrust Financial Group, Inc., an Alabama corporation, is signed to the foregoing instrument and who is known to me, acknowledged before me on this day that, being informed of the contents of the instrument, he, as such officer and with full authority, executed the same voluntarily for and as the act of said corporation.

Given under my hand this the 12^th day of October, 2007

5