FIRST AMENDMENT TO AMENDED AND RESTATED TRUST AGREEMENT BY ANDAMONG INDIAN RIVER BANKING COMPANY, as Depositor, WELLS FARGO BANK, NATIONAL ASSOCIATION, as Property Trustee,

Exhibit 10.47A

FIRST AMENDMENT TO AMENDED AND RESTATED

TRUST AGREEMENT

BY AND AMONG

INDIAN RIVER BANKING COMPANY, as Depositor,

WELLS FARGO BANK, NATIONAL ASSOCIATION, as Property Trustee,

WELLS FARGO DELAWARE TRUST COMPANY, as Resident Trustee,

THE ADMINISTRATIVE TRUSTEES NAMED HEREIN

and

ALABAMA NATIONAL BANCORPORATION

Dated as of November 19, 2003

Amended and Restated Trust Agreement

Dated as of September 30, 2002

FIRST AMENDMENT TO AMENDED AND RESTATED

TRUST AGREEMENT

THIS FIRST AMENDMENT TO AMENDED AND RESTATED TRUST AGREEMENT (this “Amendment”) is made as of this 19th day of November 2003, by and among Indian River Banking Company, a Florida corporation (“IRBC” or, before the Effective Time, the “Depositor”), as original depositor, Alabama National BanCorporation, a Delaware corporation (“ANB” or, after the Effective Time, the “Depositor”), as successor depositor, Wells Fargo Bank, National Association, a national banking association (the “Property Trustee”), as property trustee, Wells Fargo Delaware Trust Company (the “Resident Trustee”), as resident trustee, Paul A. Beindorf, Diana L. Walker and Phillip L. Tasker (each an “Administrative Trustee” and together the “Administrative Trustees”), as administrative trustees (the Property Trustee, the Resident Trustee and the Administrative Trustees referred to collectively as the “Trustees”).

WHEREAS, IRBC and the Trustees have entered into an Amended and Restated Trust Agreement, dated as of September 30, 2002 (the “Trust Agreement”), related to the issuance of preferred securities and common securities by Indian River Capital Trust I on September 30, 2002;

WHEREAS, IRBC and ANB have entered into an Agreement and Plan of Merger, dated as of October 22, 2003, that provides for the merger (the “Merger”) of IRBC with and into ANB;

WHEREAS, the Merger will become effective (a) on the date and at the time that the later of the following shall occur: (i) the Certificate of Merger reflecting the Merger shall be accepted for filing by the Secretary of State of Delaware, and (ii) the Articles of Merger reflecting the Merger shall be accepted for filing by the Secretary of State of Florida, or (b) on the date and at the time as may be otherwise specified by the parties to the Merger in the Certificate of Merger and the Articles of Merger (the time and date when the Merger becomes effective is referred to herein as the “Effective Time”);

WHEREAS, at the Effective Time, the separate corporate existence of IRBC will cease and ANB will continue as the surviving corporation following the Merger;

WHEREAS, as set forth in Section 10.6 of the Trust Agreement, the Merger is a consolidation, merger or sale involving the Depositor that is permitted under Article XII of the Indenture, and ANB desires to agree in writing to perform the Depositor’s obligations under the Trust Agreement;

WHEREAS, this Amendment has been duly authorized by all necessary action on the part of IRBC and ANB.

1

NOW, THEREFORE, in consideration of the premises set forth herein and other good and valuable consideration, IRBC, ANB and the Trustees agree as follows for the equal and ratable benefit of the Holders:

ARTICLE I

DEFINITIONS; INTERPRETATION

1.1. Definitions. Capitalized terms that are defined in the preamble or the recitals hereto shall have such meanings throughout this Amendment. Capitalized terms used but not defined in this Amendment shall have the meanings assigned thereto in the Trust Agreement. The meanings assigned to all defined terms used in this Amendment shall be equally applicable to both the singular and plural forms of such defined terms. The term “Trust Agreement” as used herein means the Amended and Restated Trust Agreement, as amended and supplemented by this Amendment, or as otherwise supplemented or amended from time to time by one or more amendments thereto or hereto entered into pursuant to the applicable provisions of the Amended and Restated Trust Agreement.

1.2. Interpretation. References in the Trust Agreement (including references in the Trust Agreement as amended or supplemented hereby) to “this Trust Agreement” (and indirect references such as “hereunder,” “herein” and “hereof”) shall be deemed references to the Trust Agreement as amended and supplemented hereby. All of the covenants, agreements and provisions of this Amendment shall be deemed to be and construed as part of the Trust Agreement to the same effect as if fully set forth therein and shall be fully enforceable in the manner provided in the Trust Agreement. Except as otherwise provided in this Amendment, all of the covenants, agreements and provisions of the Trust Agreement shall remain in full force and effect.

ARTICLE II

MERGER

Assumption by ANB. ANB, as the surviving corporation of the Merger, shall become fully responsible as of the Effective Time, without any further action, for the due and punctual performance and observance of all of the obligations of the Trust Agreement to be kept or performed by the Depositor. Upon such assumption, ANB shall succeed to and be substituted for IRBC with the same effect as if it had been named in the Trust Agreement as the original depositor, and IRBC thereupon shall be relieved of any further liability or obligation under the Trust Agreement. Upon and following the Effective Time, the parties hereto agree that all references to the “Depositor” in the Trust Agreement shall be deemed references to ANB, until a successor replaces it pursuant to the applicable provisions of the Trust Agreement and thereafter the “Depositor” shall mean such successor.

ARTICLE III

MISCELLANEOUS

3.1. Date and Time of Effectiveness. This Amendment shall become a legally effective and binding instrument at and as of the Effective Time.

2

3.2. Trust Preferred Securities Deemed Conformed. Beginning at the Effective Time, the provisions of each Trust Preferred Security then outstanding shall be deemed to be conformed, without the necessity for any reissuance or exchange of such Trust Preferred Security or any other action on the part of the Holders, IRBC, ANB or the Trustee, so as to reflect this Amendment.

3.3. Successors. All agreements of IRBC, ANB and the Trustees in this Amendment and in the Trust Agreement shall bind their respective successors.

3.4. Benefits of Amendment. Nothing in this Amendment, express or implied, shall give to any person, other than the parties hereto and their successors hereunder, any agent and the Holders, any benefit or any legal or equitable right, remedy or claim under this Amendment or the Trust Agreement.

3.5. Separability. In case any provision in this Amendment, or in the Trust Agreement, shall be invalid, illegal or unenforceable, the validity, legality and enforceability of the remaining provisions shall not in any way be affected or impaired thereby, it being intended that all of the provisions hereof shall be enforceable to the full extent permitted by law.

3.6. Trustee Responsibility. The Trustees assume no duties, responsibilities or liabilities by reason of this Amendment other than as set forth in the Trust Agreement. The Trustees assume no responsibility for the correctness of the statements herein contained, which shall be taken as statements of IRBC and ANB. This Amendment is executed and accepted by the Trustees subject to all of the terms and conditions of its acceptance of the trust under the Trust Agreement, as fully as if said terms and conditions were herein set forth in full.

3.7. Headings. The Article and Section headings of this Amendment have been inserted for convenience of reference only, are not to be considered a part of this Amendment and shall in no way modify or restrict any of the terms or provisions hereof.

3.8. Counterparts. This Amendment may be executed in counterparts, each of which shall for all purposes be deemed to be an original, and all such counterparts shall together constitute but one and the same instrument.

3.9. Governing Law. This Amendment shall be governed by and construed in accordance with the internal laws of the State of Delaware, without regard to its choice of law provisions.

[Signature Pages Follow]

3

IN WITNESS WHEREOF, the parties hereto have caused this First Amendment to Amended and Restated Trust Agreement to be duly executed, and their respective corporate seals to be hereunto affixed and attested, all as of the day and year first above written.

4

5